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Executive  Summary 

The  primary  purpose  of  the  Air  Force  Systems  Engineering  Assessment  Model  (AF 
SEAM)  is  to  promote  the  application  and  use  of  standard  Systems  Engineering  (SE) 
processes  across  the  AF  and  to  improve  the  performance  of  these  processes  through 
Continuous  Process  Improvement  (CPI).  This  is  achieved  by  providing  both  standard 
process  definitions  and  an  associated  set  of  SE  best  practices  tailored  for  use  by  United 
States  Air  Force  programs  and  projects.  These  practices  include  the  activities  performed 
by  technical  professionals  across  the  AF  charged  with  the  responsibilities  of  identifying, 
acquiring,  testing  and  sustaining  military  weapon  systems.  Combined,  these  practices 
form  the  foundation  for  SE  process  discipline  that  leads  to  repeatable  excellence  in 
product  life-cycle  management  and  higher  levels  of  customer  satisfaction.  The  processes 
and  associated  practices  address  acquirer  activities  as  well  as  activities  conducted  by  the 
integrator  or  supplier  and  other  organizations  throughout  the  supply-chain.  It  is  the 
acquirer’s  role  to  over-see  the  adequacy  of  the  SE  processes  and  ensure  effective 
implementation  of  systems  engineering.  This  includes  those  government  processes  that 
have  been  flowed  down  and  are  then  delegated  to  the  supplier.  The  final  responsibility  for 
the  performance  of  the  processes  remains  with  the  acquirer. 

AF  SEAM  was  developed  to  support  both  self-assessment  and  independent  validation  of 
systems  engineering  process  implementation.  Assessments  based  on  this  model  should 
address  the  practices  applicable  to  the  area  under  examination  for  both  those  areas 
carried  out  primarily  by  the  acquirer  and  the  acquirer’s  SE  processes  intended  to  provide 
insight/oversight  of  others.  Therefore,  this  model  is  intended  to  be  tailored  to  meet  the 
needs  of  the  user. 

Many  of  the  best  practices  contained  in  AF  SEAM  were  derived  from  various  Software 
Engineering  Institute  (SEI)  /  Carnegie  Mellon,  Capability  Maturity  Model  Integration® 
(CMMI®)  products.  Additionally,  various  international  and  industry  standards,  Department 
of  Defense  (DoD)  publications  and  development  team  members’  expert  knowledge 
significantly  contributed  to  the  material  contained  in  this  model.  It  is  essential  to  note  that 
AF  SEAM  is  a  process  assessment  tool  which  is  designed  to  assess  the  presence  of 
needed  SE  processes  as  a  “leading  indicator”  to  subsequent  delivery  success.  While  the 
tool  assesses  the  existence  of  SE  process  work  products  (i.e.  CONOPS,  plans,  technical 
documents,  etc)  it  does  not  assess  the  outcomes  delivered  to  the  customer.  The  model 
concentrates  on  “what”  SE  processes  must  be  in  place  which,  when  properly  executed, 
increase  the  likelihood  customer  needs  will  be  satisfied.  Therefore,  AF  SEAM  must  be 
used  in  conjunction  with  other  more  traditional  tools  to  gain  a  full  understanding  of  project 
/  program  status.  Also,  the  model  was  designed  to  be  flexible  and  simultaneously  take 
full  advantage  of  creative  solutions  and  CPI  by  not  bounding  the  user  to  prescribed 
implementation  approaches  (“how  to”)  for  achieving  systems  engineering  best  practices. 

Finally,  in  its  present  state  of  development  AF  SEAM  is  not  100%  correct  and  itself  relies 
upon  CPI.  Recommended  improvements  or  comments  should  be  directed  to  your 
center’s  AF  SEAM  POC. 
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1.  Introduction 

The  lack  of  robust  systems  engineering  processes  has  been  cited  as  one  of  the  root 
causes  of  negative  occurrences  in  acquisition  and  sustainment  programs  across  the 
Department  of  Defense.  Congressional  and  Department  of  Defense  guidance 
emphasizes  the  need  for  sustained  disciplined  processes  and  process  improvement 
including  the  measurement  of  process  performance.  The  goal  of  this  guidance  is  to 
influence  the  outcome  of  the  acquisition  process  and  delivery  of  the  right  capabilities  to 
operational  users,  on  schedule,  and  at  predictable  costs.  At  the  same  time,  leaders  in  the 
Services  continue  to  face  aggressive  performance,  cost,  and  schedule  baselines. 

One  way  to  meet  these  challenges  is  through  the  disciplined  application  of  systems 
engineering,  which  provides  the  technical  framework  to  enable  sound  decision  making 
relative  to  trade  studies  among  performance,  supportability,  cost,  and  schedule  while 
considering  risk.  The  successful  implementation  of  proven,  disciplined  systems 
engineering  processes  results  in  a  total  capability  solution  that  is: 

•  Robust  to  changing  technical,  manufacturing,  and  operating  environments; 

•  Adaptive  to  the  needs  of  the  user; 

•  Balanced  among  the  multiple  requirements,  design  considerations,  design 
constraints,  and  project  schedules  and  budgets;  and 

•  Operates  smoothly  in  a  complex  system-of-systems  environment  when  required. 

Applying  this  approach  demands  renewed  dedication  to  defining,  implementing, 
measuring,  and  maintaining  the  systems  engineering  processes  that  are  fundamental  to 
successfully  delivering  a  technically  sound  capability. 

2.  Purpose 

The  primary  purpose  of  the  Air  Force  Systems  Engineering  Assessment  Model  (AF 
SEAM)  is  to  promote  the  application  and  use  of  standard  Systems  Engineering  (SE) 
processes  across  the  AF  and  to  improve  the  performance  of  these  processes  through 
Continuous  Process  Improvement  (CPI).  AF  SEAM  was  developed  to  support  both  self- 
assessment  and  independent  validation  of  systems  engineering  process  implementation. 
Assessments  based  on  this  model  should  address  the  practices  applicable  to  the  area 
under  examination  for  both  those  areas  carried  out  primarily  by  the  acquirer  and  the 
acquirer’s  SE  processes  intended  to  provide  insight/oversight  of  others.  AF  SEAM  is 
lean  by  design  and  is  therefore  targeted  to  meet  the  users  need  when  a  more 
complex/detailed  approach  (e.g.  CMMI)  is  not  required. 

This  model  defines  a  standard  view  of  systems  engineering  processes  within  the  AF  and 
defines  expected  SE  process  outcomes.  The  descriptive  guidance  under  the  title  “Other 
Considerations”  for  each  practice  provides  suggestions  for  implementation  that  can  be 
used  for  training  and  continuous  process  improvement.  While  this  model  identifies 
practices  that  should  be  implemented  it  does  not  prescribe  specific  implementation 
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approaches.  This  provides  a  basis  for  acquisition  process  discipline,  while  balancing  the 
need  for  flexibility,  and  tailoring  to  specific  AF  user  needs. 

3.  AF  SEAM  Model  Content 

AF  SEAM  defines  ten  AF  standard  SE  process  areas,  lists  associated  goals  under  each 
process  area  and  provides  associated  specific  and  generic  practices.  Many  of  the  best 
practices  contained  in  AF  SEAM  were  derived  from  various  Software  Engineering 
Institute  (SEI)  /  Carnegie  Mellon,  Capability  Maturity  Model  Integration®  (CMMI®) 
products.  Additionally,  various  international  and  industry  standards,  Department  of 
Defense  publications  and  development  team  members’  expert  knowledge  significantly 
contributed  to  the  material  contained  in  this  model.  It  is  essential  to  note  that  AF  SEAM 
is  a  process  assessment  tool  which  is  designed  to  assess  the  presence  of  needed  SE 
processes  as  a  “leading  indicator”  to  subsequent  delivery  success.  While  the  tool 
assesses  the  existence  of  SE  process  work  products  (i.e.  CONOPS,  plans,  technical 
documents,  etc)  it  does  not  assess  the  outcomes  delivered  to  the  customer.  The  model 
concentrates  on  “what”  SE  processes  must  be  in  place  which,  when  properly  executed, 
increase  the  likelihood  customer  needs  will  be  satisfied.  This  is  due  to  the  fact  that  the 
quality  of  a  System  or  Product  is  highly  influenced  by  the  quality  of  the  process  used  to 
develop  and  maintain  it. 

AF  SEAM  was  designed  to  facilitate  user  tailoring.  Tailoring  is  accomplished  in  one  of 
two  ways.  First,  specific  practices  which  do  not  apply  to  the  area  under  examination  may 
be  coded  not  applicable  (N/A),  and  therefore  would  not  be  assessed.  Care  should  be 
taken  to  ensure  a  practice  is  not  omitted  which  upon  closer  examination  may  not  be 
customarily  considered  for  inclusion,  but  should  be  included  to  promote  overall  mission 
success.  Generic  practices  may  not  be  omitted,  although  in  the  highly  unlikely  event  that 
an  entire  process  area  is  omitted,  generic  practices  would  not  apply  to  that  area. 
Secondly,  users  may  add  to  AF  SEAM  to  cover  areas  not  specifically  included  in  the  base 
model.  Where  tailoring  is  used,  content  may  be  added  to  the  AF  SEAM,  however, 
original  content  may  not  be  deleted. 

3.1.  AF  SEAM  Core  Document  Format 

AF  SEAM  is  built  around  ten  Process  Areas  which  are  at  the  highest  level  of  the  model. 
These  process  areas  are  supported  by  Goals.  Goals  are  amplified  by  Practices. 
Practices  fall  into  two  categories:  Specific  and  Generic.  Following  is  a  detailed 
description  of  each. 

3.1.1.  Process  Areas  (PAs) 

Process  areas  are  individually  described  in  terms  that  define  the  overarching  purpose 
and  concepts  associated  with  the  process  area.  A  process  area  is  further  defined  by  a 
grouping  of  related  goals  and  practices  which  implemented  collectively  satisfy  the  stated 
purpose  of  the  process  area.  It  should  also  be  noted  that  multiple  individual  processes 
are  required  to  successfully  achieve  practices  and  goals  which  comprise  an  overall 
process  area. 
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AF  SEAM  is  comprised  of  ten  process  areas  which  are  presented  in  alphabetical 
order  with  their  associated  acronym  below: 

CM:  Configuration  Management 

DA:  Decision  Analysis 

D:  Design 

M:  Manufacturing 

PP:  Project  Planning 

R:  Requirements 

RM:  Risk  Management 

TFS:  Transition,  Fielding,  &  Sustainment 

TMC:  Technical  Management  &  Control 

V:  Verification  &  Validation 

3.1.2.  Specific  Goals  (SG) 

A  specific  goal  describes  the  unique  characteristics  that  must  be  present  to  satisfy  the 
goal.  A  specific  goal  is  a  required  model  component  and  is  helpful  in  the  grouping  of 
associated  practices  and  is  used  in  assessments  to  improve  clarity  of  understanding. 

3.1.3.  Specific  Practices  (SP) 

The  title  associated  with  each  specific  practice  defines  the  desired  activity.  Below  the 
practice  title  is  an  area  titled  “Description”.  The  description  area  contains  amplification 
and  clarification  of  the  practice.  In  most  cases  the  description  defines  the  minimal  activity 
required  to  successfully  meet  the  practice. 

3.1.4.  Generic  Practices 

Generic  practices  are  called  generic  because  the  same  practice  applies  to  all  ten  process 
areas  individually.  A  generic  practice  is  the  description  of  an  activity  which  is  considered 
important  to  facilitate  successful  achievement  of  the  specific  practices  and  process  area 
goals  associated  with  an  overarching  process  area. 
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Process  areas  and  their  associated  number  of  goals  and  practices  are  summarized  in  the 
table  below: 


Specific 

Generic 

Total 

Process  Area 

Goals 

Practices 

Practices 

Practices 

Configuration  Mgmt 

3 

8 

7 

16 

Decision  Analysis 

1 

5 

7 

12 

Design 

3 

14 

7 

21 

Manufacturing 

4 

12 

7 

19 

Project  Planning 

3 

15 

7 

22 

Requirements 

4 

13 

7 

21 

Risk  Mgmt 

3 

7 

7 

14 

Trans,  Fielding,  &  Sus 

4 

15 

7 

22 

Tech  Mgmt  &  Control 

4 

15 

7 

18 

Verification  &  Validation 

5 

16 

7 

23 

Total 

34 

120 

70 

190 

3.1.5.  Typical  Work  Products 

The  typical  work  products  section  lists  various  outputs  which  are  expected  to  be  an 
outcome  of  the  process  /  processes  which  are  executed  to  successfully  achieve  the 
practice.  These  examples  are  called  “typical  work  products”  because  while  they  are 
often  the  outcomes  produced,  other  work  products  that  are  just  as  effective  may  also 
be  produced  but  are  not  listed. 

3.1.6.  Reference  Material 

Reference  material  has  been  provided  for  practices  under  the  heading  “Reference 
Material”.  The  items  listed  are  provided  as  an  aid  and  are  not  intended  to  represent 
all  potentially  applicable  reference  materials.  In  addition  to  the  tailoring  of  practices 
previously  described  under  paragraph  2.0,  user  tailoring  which  includes  the  addition  of 
applicable  lower  level  reference  material  to  this  section  is  encouraged.  Users  should 
not  rely  upon  the  information  given  in  this  model  alone  and  are  therefore  are 
encouraged  to  become  familiar  with  referenced  documents.  Most  reference  material 
has  been  hyperlinked  to  common  access  sites  that  do  not  require  a  Government 
Common  Access  card  (CAC).  To  access  the  links,  simply  place  the  cursor  over  the 
reference  and  perform  the  <Control><Click>  sequence. 

3.1.7.  Other  Considerations 

Other  considerations  are  detailed  descriptions  that  provide  guidance  for  interpreting 
and  implementing  a  practice.  These  considerations  suggest  “best  practices”  within  a 
practice  area.  In  addition  to  the  tailoring  of  practices  and  reference  materials 
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previously  described,  user  tailoring  which  includes  the  addition  of  lessons  learned  is 
encouraged. 


3.2.  Systems  Engineering  References 

Many  systems  engineering  process  standards  and  models  exist  that  describe  best 
practices  in  accomplishing  systems  engineering.  Process  areas,  goals  and  associated 
practices  described  in  this  model  were  derived  from  a  number  of  sources,  including  those 
listed  below.  Additionally,  proven  practices  have  also  been  gleaned  from  experienced 
professionals  who  assisted  with  the  model’s  development.  Examples  of  the  references 
used  include  the  following: 

•  ISO/IEC  15288,  Systems  Engineering-System  Life  Cycle  Processes 

•  ANSI/EIA  632,  Processes  for  Engineering  a  System 

•  IEEE  1220,  Application  and  Management  of  the  Systems  Engineering  Process 

•  EIA  731 ,  Systems  Engineering  Capability  Model 

•  CMMI,  Capability  Maturity  Model  Integration 

•  Defense  Acquisition  Guidebook,  Chapter  4 

•  AFI  63-1201,  Life  Cycle  Systems  Engineering 

•  IEEE/EIA  12207,  Software  Life  Cycle  Processes 

•  Air  Force  Weapon  System  Software  Management  Guidebook 
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Process  areas  and  associated  definitions  were  developed  to  best  meet  AF  SE  needs  and 
are  in  alignment  with  current  DoD  and  AF  guidance  as  summarized  in  the  table  below: 


AF  SEAM 

Defense  Acquisition  Guide 

AFI  63-1201 

Requirements 

Reqts  Analysis,  Reqts  Mgmt, 
Stakeholder  Reqts  Definition 

Reqt  Dev  &  Mgmt,  & 
Architecture 

Design 

Architectural  Design, 
Integration  &  Interface 
Mgmt 

Design  &  Interface  Mgmt 

Verification  &  Validation 

Verification  &  Validation 

Test  &  Evaluation, 
Verification  &  Validation 

Manufacturing 

Implementation 

Design 

Transition,  Fielding,  & 
Sustainment 

Transition 

Design 

Project  Planning 

Technical  Planning 

Planning 

Configuration 

Management 

CM,  Data  Mgmt,  Technical 
Data  Mgmt 

Configuration  Mgmt,  Data 
Mgmt 

Risk  Management 

Risk  Mgmt 

Integrated  Risk  Management 

Technical  Mgmt  & 
Control  (PMC) 

Technical  Assessment 

Technical  Reviews  & 
Measurements 

Decision  Analysis 

Decision  Analysis 

Decision  Analysis 

3.3.  Terminology 

In  general,  this  model  uses  the  terminology  used  within  the  CMMI  framework.  References 
to  CMMI  and  AF  SEAM  terminology  should  be  understood  in  both  the  acquisition  and  the 
process  improvement  contexts. 

•  The  term  “project”  denotes  a  managed  set  of  interrelated  resources  that  delivers 
one  or  more  products  to  a  customer  or  end-user.  In  this  document,  a  “project” 
could  refer  to  an  entire  acquisition  program  or,  perhaps,  a  major  subset  of  an 
acquisition  program.  The  scope  of  the  term  is  tailored  to  the  specific  acquisition. 

•  The  term  “organization”  is  typically  used  to  denote  an  administrative  structure  in 
which  people  collectively  manage  one  or  more  projects. 
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•  The  term  “acquirer”  refers  to  the  organization  responsible  for  acquisition  of  the 
product  or  component.  This  is  typically  the  Government  and  might  include  both  the 
project  or  program  office  and  sponsoring  office.  It  can  also  apply  recursively  down 
the  supply-chain  (i.e.,  the  integrator  fulfills  an  acquirer  role  when  working  with  sub¬ 
tiers  in  the  supply-chain) 

•  The  term  “integrator”  refers  to  the  organization  performing  the  work  of  product 
integration  (e.g.,  the  prime  contractor). 

•  The  term  “supplier”  refers  to  all  organizations  in  the  supply-chain  including  the 
integrator  and  other  organizations  responsible  for  providing  a  subassembly, 
component  or  part  of  the  product. 

•  The  term  “work  product”  is  any  artifact  produced  by  a  process. 

•  The  term  “product”  denotes  a  tangible  output  or  service  that  is  a  result  of  a 
process  and  that  is  intended  for  delivery  to  a  customer  or  end  user.  A  product  is  a 
work  product  that  is  delivered  to  the  customer. 

•  The  term  “establishing”  includes  changing  documenting  and  providing  a 
description. 

•  The  term  “maintaining”  includes  changing  documentation  as  necessary. 

•  The  term  “system”  is  not  used  in  the  CMMI  framework  or  in  the  AF  SEAM  process 
areas  except  in  the  general  sense  (e.g.,  systems  engineering). 

•  The  term  “systems  engineering”  includes  software  engineering. 

•  The  term  “stakeholder”  refers  to  a  person,  group,  or  organization  that  affects  or 
can  be  affected  by  an  organization’s  actions. 

•  Readers  should  decide  how  these  terms  apply  to  their  environment. 
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4.  AF  SEAM  Assessment  Process 

The  below  diagram  describes  the  overall  AF  SEAM  assessment  process.  The  following 
paragraphs  summarize  what  occurs  during  each  phase  of  the  process. 


4.1.  Leadership  Engagement  -  A 

The  process  is  started  by  leadership  who  direct  that  a  project/program  office  conduct  a 
self-assessment  (A).  Leadership  initiates  a  self-assessment  based  upon  a  multitude  of 
factors  including:  an  event,  schedule,  Earned  Value  Management  (EVM),  other 
feedback,  type  of  program,  etc.  At  the  outset,  leadership  may  also  direct  that  the  self- 
assessment  be  followed  by  a  validation  assessment,  or  may  choose  to  wait  until  the 
results  of  the  self-assessment  are  known 

4.2.  Transitioning  From  Start  (A)  To  Self-Assessment  (B) 

Following  leadership  direction  to  start  the  process,  the  Center  Engineering  home  office, 
or  others  directed  by  leadership  who  are  knowledgeable  of  the  AF  SEAM  assessment 
process,  engages  with  the  project/program  selected  for  study.  If  the  people  working  in 
the  subject  project/program  office  have  previously  conducted  AF  SEAM  self-assessments 
and  all  participants  are  familiar  with  the  assessment  process,  roles  and  responsibilities, 
recording  and  reporting  requirements,  then  no  further  action  on  the  part  of  the 
Engineering  home  office  is  required.  However,  if  the  individuals  working  within  the  area 
of  study  are  unfamiliar  with  AF  SEAM,  then  the  Engineering  home  office  initiates  actions 
to  provide  project/program  leadership  with  the  AF  SEAM  Leadership  Training.  Members 
of  the  Engineering  home  office  will  assist  with  the  identification  of  specific  roles  and 
responsibilities,  planning  for  the  overall  assessment  effort,  and  providing  AF  SEAM  Self- 
Assessment  Training  to  all  self  assessors  needing  the  training. 

4.3.  Self  Assessment  Phase  -  B 

During  the  self-assessment  phase,  members  of  the  selected  project/program  office  will 
perform  a  self-assessment.  Before  initiating  the  self-assessment,  AF  SEAM  will  be 
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tailored  to  meet  the  specific  needs  of  that  particular  project/program.  Tailoring  will  be 
accomplished  as  described  in  paragraph  3,  AF  SEAM  Model  Content.  The  self- 
assessment  is  accomplished  using  the  AF  SEAM  Assessment  Tool  or  other  locally 
approved  recording  mechanism.  Every  practice  not  specifically  omitted,  and  practices 
added  during  tailoring,  will  be  individually  assessed  and  graded.  As  the  self  assessors 
complete  the  self-assessment,  Engineering  home  office  personnel  will  be  available  to 
answer  questions  and  provide  assistance  as  required.  At  the  conclusion  of  the  self- 
assessment,  results  are  reported  to  the  project/program  leadership,  and  when  requested 
also  to  those  who  initiated  the  process  if  they  are  different.  (See  paragraph  6,  Results 
Reporting). 

4.4.  Validation  Determination  -  C 

If  not  already  directed  to  do  so,  at  this  time  leadership  will  determine  if  an  independent 
validation  is  required.  If  a  validation  assessment  is  not  desired  then  the  results  are 
posted  to  file  and  the  process  terminates.  However,  if  leadership  determines  that  a 
validation  assessment  is  required,  the  Engineering  home  offices  (or  others  designated  as 
responsible)  are  notified. 

4.5.  Transitioning  From  Validation  Determination  (C)  To  Validation  Assessment  (D) 

Following  completion  of  the  self-assessment  phase,  if  leadership  has  directed  that  a 
validation  assessment  be  accomplished,  actions  are  initiated  to  prepare  for  an 
independent  validation.  Upon  receipt  of  this  direction  the  Center  Engineering  home 
office,  or  others  directed  by  leadership  who  are  knowledgeable  of  the  AF  SEAM 
assessment  process,  will  begin/continue  the  assessment  planning  process.  Planning  will 
encompass  all  actions  necessary  to  accomplish  a  validation  assessment.  This 
traditionally  includes:  identifying  and  securing  team  members,  training  validation 
assessment  team  members,  arranging  all  necessary  logistics  support,  planning 
interviews,  coordinating  with  leadership  to  establish  a  schedule,  securing  data  from  the 
self  assessment  phase,  and  defining  team  member  roles  and  responsibilities.  It  is 
customary  that  validation  teams  be  comprised  of  members  who  were  not  previously 
involved  in  the  self-assessment,  and  whenever  possible  members  will  not  be  from  the 
same  organization. 

4.6.  Validation  Assessment  Phase  -  D 

The  validation  assessment  phase  begins  with  a  team  meeting  during  which  the  team 
leader  reviews  the  information  prepared  in  paragraph  4.5.,  Transitioning  From  Validation 
Determination  (C)  To  Validation  Assessment  (D).  Assessment  team  members  should 
leave  this  meeting  with  a  clear  understanding  of  the  effort  before  them,  their  individual 
role  and  responsibilities,  the  resources  available,  and  the  schedule.  Assessment  team 
members  begin  their  individual  work  by  reviewing  all  available  material  for  their  area  of 
assignment  including  the  self-assessment  data.  The  information  derived  during  the  self- 
assessment  serves  as  a  starting  point  from  which  independent  assessors  determine 
correctness  of  model  usage  during  the  self-assessment  phase.  Validation  assessors 
also  use  this  data  in  combination  with  their  interviews  to  confirm  correct  interpretation  of 
practices  by  the  self  assessors.  Independent  assessors  provide  accurate  feedback  and 
recommendations  to  project/program  personnel.  Although  the  details  of  how  this  is 
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accomplished  are  determined  at  the  Center  level  (or  Wing  level  in  the  absence  of  a 
Center),  it  is  intended  that  the  validation  assessment  be  a  collaborative  effort  between 
the  project  personnel  and  the  validation  assessment  team.  AF  SEAM  is  not  intended  to 
be  a  compliance  check  or  an  audit  of  the  project/program.  Instead  it  is  intended  to 
facilitate  a  mutual  understanding  of  ways  to  obtain  process  excellence  as  described  in  AF 
SEAM,  identify  gaps  in  specific  and  generic  practice  attainment  with  a  mindset  toward 
continuous  process  improvement.  Areas  that  are  identified  as  needing  improvement 
should  attract  increased  attention  and  support  from  leadership  who  aid  in  closing  the 
process  performance  gaps.  At  the  conclusion  of  the  validation  assessment,  actionable 
recommendations  are  provided  by  the  validation  assessment  team  to  project/program 
members  and  leadership.  It  is  suggested  that  project/program  personnel  then  prioritize 
and  address  the  gaps  based  on  their  business  and  process  improvement  goals  and 
objectives. 

4.7.  Posting  Results  -  E 

Information  is  compiled  as  directed  by  local  policy  and  procedures.  Results  must  be 
tabulated  and  formatted  to  support  standard  reporting  as  defined  in  paragraph  6,  Results 
Reporting.  Additionally,  results  should  be  maintained  in  a  manner  that  facilitates 
feedback  and  thereby  supports  continuous  process  improvement.  Finally,  results  must 
be  readily  accessible  by  project/program  personnel  to  enable  self-improvement. 


4.8.  Feedback 

The  purpose  of  feedback  is  primarily  three-fold.  First  and  foremost  AF  SEAM  is  a 
process  improvement  tool  that  is  intended  to  be  employed  by  process  owners,  those 
working  within  these  processes,  and  those  directly  overseeing  them.  For  this  tool  to  be 
employed  successfully,  it  relies  upon  the  candor  of  process  participants  in  an 
environment  of  leadership  support  to  improve  the  process.  Therefore,  the  primary  use  of 
AF  SEAM  is  by  process  participants  for  the  purpose  of  aiding  all  to  better  improve  the 
process  and  its  associated  products/outcomes.  The  second  primary  use  of  AF  SEAM  is 
the  capture  of  process  improvement  ideas  and  lessons  learned  in  an  area  that  will  be 
immediately  visible  to  those  needing  this  information.  This  is  accomplished  by  updating 
the  area  under  each  practice  titled  “Other  Considerations”.  Center  Engineering  home 
offices  may  update  this  area  for  the  AF  SEAM  tool  being  used  at  their  Center  as  part  of 
the  tailoring  process.  Centers  are  also  encouraged  to  share  this  information  with  the 
office  responsible  for  AF  SEAM  configuration  management  and  control,  which  after 
reviewing  the  input  with  other  stakeholders  across  the  AF,  may  elect  to  include  this 
additional  information  in  future  model  iterations.  The  third  primary  use  of  AF  SEAM 
feedback  is  to  share  results  with  those  who  are  best  positioned  to  facilitate  improvement 
above  the  project/program  level  (see  paragraph  6,  Results  Reporting).  Leadership 
assistance  may  include:  provision  of  additional  resources,  policy  change,  or  mutual 
acceptance  of  risk. 

5.  Assessment  Methodology 

5.1.  Self-Assessment  Methodology 
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The  Air  Force  SEAM  relies  on  a  combination  of  various  assessment  techniques. 
Participants  perform  self-assessments  using  information  and  data  readily  available  to 
them.  Self-assessors  may  use  the  tool  provided  in  the  AF  SEAM  Tool  Suite  or  one  that  is 
made  available  through  the  Center  organization  to  which  they  are  assigned.  Although 
there  is  flexibility  in  recording  tool  usage,  results  reporting  is  standardized.  Each 
practice  ( specific  and  generic)  are  graded  as  either  satisfied  (1  or  colored  green)  or  not 
satisfied  (0  and  colored  red). 

5.2.  Validation-Assessment  Methodology 

Validation  assessment  teams  are  usually  comprised  of  individuals  from  the  parent 
organization’s  Engineering  home  office  and  others  selected  by  this  office  to  augment  the 
team.  Assessors  rely  upon  the  information  developed  during  the  self-assessment  phase. 
Validation  assessors  use  this  information,  personal  interviews,  and  independent  research 
to  determine  if  a  practice  is  being  met.  Those  directly  responsible  for  the  processes 
under  consideration  are  a  critical  resource  to  the  validation  assessors.  As  with  self 
assessors,  validation  assessors  may  use  the  tool  provided  in  the  AF  SEAM  Tool  Suite  or 
one  that  is  made  available  through  the  Center  organization  to  which  they  are  assigned. 
Although  there  is  flexibility  in  recording  tool  usage,  results  reporting  is  standardized. 
Each  practice  ( specific  and  generic)  are  graded  as  either  satisfied  (1  or  colored  green)  or 
not  satisfied  (0  and  colored  red). 

6.  Results  Reporting 

6.1.  Overview 

Results  reporting  takes  into  consideration  the  needs  of  those  to  whom  the  information  is 
being  reported.  The  goal  is  to  provide  various  levels  of  organizational  leadership  with: 

•  the  information  needed,  and 

•  in  the  level  of  detail  required, 

to  best  support  continued  process  improvement.  For  this  reason,  Program  Managers 
(PMs)  and  members  of  the  program/project  office  using  AF  SEAM  need  to  see  the  results 
down  to  the  individual  practice  level.  PMs  need  this  level  of  understanding  regarding  the 
various  SE  processes  operating  within  their  individual  projects/programs  to  know  where 
to  focus  their  attention  down  to  the  Action  Officer  level.  Center  and  MAJCOM  leadership 
need  the  information  in  a  format  that  best  enables  them  to  view  entire  portfolios  at  the 
process  level.  This  is  to  facilitate  Center  and  MAJCOM  level  analysis  in  a  broader  sense. 
This  broader  process  view  allows  Center  and  MAJCOM  leaders  to  focus  on  continually 
improving  processes  across  entire  Centers/MAJCOM’s  while  simultaneously  allowing 
individual  PMs  to  focus  on  the  details  of  their  individual  areas  of  responsibility. 
Therefore,  while  the  underlying  data  is  the  same,  the  outputs  are  tailored  to  meet  the 
needs  of  those  using  it. 
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Reporting  is  accomplished  for  specific  and  generic  practices  separately.  This  is  to 
provide  appropriate  visibility  into  the  separate  practices,  and  to  avoid  confusion  by  the 
reviewer  due  to  different  underlying  scoring  algorithms.  Each  practice  is  graded  as  either 
satisfied  or  not  satisfied.  These  results  are  then  aggregated  to  yield  a  color  as  indicated 
in  the  table  below. 


Color 

Specific  Practice  (%) 

Generic  Practice  (#) 

Green 

90-100 

6-7 

Yellow 

65-89 

4-5 

Red 

0-64 

0-3 
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The  formats  provided  below  are  recommended  and  viewed  as  essential  to  promote 
standardization  across  the  AF.  The  information  contained  in  each  is  notional.  Additional 
views  may  be  added  as  leadership  determines  necessary. 


6.2.  Project/Program  Level  Recommend  Reporting  Format 
Specific  Practices 

Percentage 
(of  those 
practices 
scored) 

Specific  Goal  1 
SP  1.1 
SP  1  .2 
SP  1  .3 
SP  1  .4 
SP  1 .5 


Specific  Goal  2 

SP  2.1 
SP  2.2 
SP  2.3 
SP  2.4 
SP  2.5 
SP  2.6 
SP  2  7 
SP  2.8 

Specfic  Goal  3 
SP  3.1 
SP  3.2 
SP  3.3 
SP  3.4 
SP  3.5 

Specific  Goal  4 
SP  4.1 
SP  4.2 
SP  4.3 
SP  4.4 
SP  4.5 

Specific  Goal  5 
SP  5.1 


~T  j  T _ 1  i  ~ ■  _  i 

i _ i  i  ■  ■  i 

™  I  1=1 


_lU  :  1 :  KJ 


SP  5.2 


N/A 


N/A 


Generic  Practices 


PA/GP 

GP1 

GP2 

GP3 

GP4 

GP5 

GP6 

GP7 

GP  Overall 

CM 

1 

1 

1 

1 

1 

1 

1 

7 

DA 

1 

1 

1 

1 

1 

5 

D 

1 

1 

1 

1 

1 

1 

1 

7 

M 

1 

1 

1 

1 

1 

1 

1 

7 

PP 

1 

1 

1 

1 

4 

R 

1 

1 

1 

1 

1 

1 

1 

7 

RM 

1 

1 

1 

1 

1 

1 

0  6 

S 

1 

1 

1 

1 

1 

1 

1 

7 

TMC 

1 

1 

1 

1 

1 

1 

6 

V 

1 

1 

1 

1 

1 

1 

1 

7 
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6.3.  Center  &  MAJCOM  Level  Recommend  Reporting  Format 

Specific  Practices  Roll  Up  By  Process  Area 

Process  Area  Assessment  Results 
XXX  Center 


o. 

«+- 

o 

-a 

E 

3 


Generic  Practices  Roll  Up  By  Process  Area 


Generic  Practice  Assessment  Results 
XXX  Center 


GPl  GP2  GP3  GP4  GP5  GP6  GP7 

Process  Area 


7.  Training 

There  are  three  different  training  modules  that  have  been  specifically  designed  to  support 
AF  SEAM:  Leadership,  Self-Assessment,  and  Independent  Validation  Training.  Training 
is  provided  as  part  of  the  AF  SEAM  Tool  Suite. 
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7.1.  AF  SEAM  Leadership  Training 

Leadership  training  is  designed  to  take  approximately  one  hour  and  is  specifically 
designed  for  members  of  leadership/management  who  oversee  projects/programs,  or 
have  responsibility  for  their  oversight. 

7.2.  AF  SEAM  Self-Assessment  Training 

AF  SEAM  Self-Assessment  training  is  specifically  designed  for  those  who  will  be 
performing  a  self-assessment  using  the  AF  SEAM.  This  training  introduces  the 
participant  to  AF  SEAM,  the  assessment  process,  the  details  of  its  construction,  the  basic 
principles  of  continuous  process  improvement,  and  the  importance  of  their  role  as  a 
process  participant.  It  also  outlines  the  goals  of  AF  SEAM  and  demonstrates  results 
reporting  in  that  context.  The  participant  is  taken  through  various  exercises  to  provide 
advance  “hands-on”  training  to  assist  in  becoming  comfortable  with  using  AF  SEAM,  and 
to  simultaneously  promote  standardization  of  reporting.  Training  is  designed  to  take 
approximately  4  hours,  is  provided  in  person  by  an  individual  who  is  intimately  familiar 
with  and  experienced  in  the  use  of  AF  SEAM,  and  is  intended  to  be  given  “just-in-time”. 

7.3.  AF  SEAM  Validation  Assessment  Team  Training 

AF  SEAM  Validation-Assessment  training  is  specifically  designed  for  those  who  will  be 
members  of  an  AF  SEAM  validation  assessment  team.  In  addition  to  briefly  covering  the 
materials  contained  in  the  self-assessment  training,  emphasis  will  be  on  their  specific 
roles  and  responsibilities  as  they  relate  to  the  assigned  task.  Therefore,  the  training  is 
tailored  to  meet  the  specific  needs  of  team  members  for  the  validation  assessment  being 
undertaken.  Training  is  designed  to  take  approximately  4  hours.  Whenever  possible,  the 
team  lead  will  personally  conduct  this  training.  If  the  team  lead  is  not  available,  then  the 
training  is  provided  in  person  by  an  individual  who  is  intimately  familiar  with  and 
experienced  in  the  use  of  AF  SEAM.  As  with  the  self-assessment  training,  validation 
assessment  team  training  is  intended  to  be  given  “just-in-time”. 
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8.  Process  Areas 

8.1  Configuration  Management  (CM) 

The  purpose  of  Configuration  Management  is  to  establish  and  maintain  the  integrity  of 
the  product’s  technical  baseline  while  accommodating  change  and  providing  a  clear, 
concise,  and  valid  description  of  the  product  to  concerned  parties. 

Configuration  management  is  the  process  of  managing  products,  facilities,  and  processes 
by  managing  the  information  about  them,  including  changes,  and  ensuring  they  are  what 
they  are  supposed  to  be  in  every  case. 

A  “configuration  item”  is  defined  as  an  aggregation  of  work  products  that  is  designated  for 
configuration  management  and  treated  as  a  single  entity  within  the  configuration 
management  process. 

A  “baseline”  is  set  of  specifications  or  work  products  that  has  been  formally  reviewed  and 
agreed  on,  that  thereafter  serves  as  the  basis  for  further  development  and  authoritative 
representation  of  the  product. 

A  progression  of  technical  baselines  is  developed  during  the  development  life-cycle  of  a 
product.  Once  the  baseline  is  established,  changes  to  the  items  can  only  be  done 
through  a  formal  change  process.  Baselines  provide  a  stable  basis  for  continuing 
evolution  of  configuration  items.  An  example  of  a  baseline  is  an  approved  description  of  a 
product  that  includes  internally  consistent  versions  of  requirements,  requirement 
traceability  matrices,  designs,  and  end-user  and  support  documentation. 

The  Configuration  Management  process  area  can  best  be  described  by  the  following 
goals  and  practices. 

CMG1  The  approach  for  technical  baseline  management  is  defined  and 
documented. 

CMG1  PI  Identify  accountability  for  the  disposition  of,  access  to,  release  of  and 
control  of  the  technical  baselines. 


Description:  Establish  and  maintain  a  strategy  for  technical  baseline  management.  Specify  which 
configuration  items  will  be  controlled  during  the  various  phases  of  product  development  i.e. ,  what  baselines 
will  be  established  and  when.  Define  the  membership  of  the  configuration  control  boards.  Define  which 
members  will  have  authority  to  access  and  release  design  and  product  baselines,  and  authorize  changes  to 
the  baselines  as  the  product  progresses  through  its  life  cycle. 

Typical  Work  Products: 


□  Systems  Engineering  Plan 

□  Configuration  Control  Board  Charter 


Reference  Material:  DI-CMAN-80858  B, 
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Other  Considerations: 

CMG1P2  Establish  and  maintain  plans  for  managing  the  configuration  of  the 
product. 


Description:  Document  plans  and  processes  for  technical  baseline  management  in  appropriate 
documents.  Describe  how  consistency  between  the  product  definition,  the  product’s  physical  and  functional 
configuration,  and  the  CM  records  is  achieved  and  maintained  throughout  the  product’s  life  cycle. 


Typical  Work  Products: 


□  Systems  Engineering  Plan 

□  Configuration  Management  Plan 


Reference  Material: 

Other  Considerations:  Consider  the  project’s  related  needs  for  management  of  data  and  interfaces. 
Configuration  management  addresses  obsolescence  and  technology  refreshment.  Change  management 
process  includes  a  checklist  for  all  changes  to  be  evaluated  for  impact  to  the  OSS&E  Baseline  Document. 
Configuration  management  of  products  may  be  documented  in  the  Systems  Engineering  Plan  or  the 
Configuration  Management  Plan. 


CMG2  Establish  and  maintain  technical  baselines  while  managing  change. 

CMG2P1  Identify  the  configuration  items  and  related  work  products  that  will  be 
placed  under  configuration  management. 


Description:  Select  the  configuration  items  and  work  products  that  define  them.  Assign  unique  identifiers 
to  each  configuration  item.  Identify  the  important  functional  and  physical  attributes.  Specify  control 
characteristics  for  each  item  (e.g.,  organization  responsible,  degrees  of  control  and  when  the  item  should 
be  placed  under  configuration  management). 

Typical  Work  Products: 


□  Configuration  Management  Plan 

□  Configuration  Items  (list,  table,  spreadsheet  etc.) 


Reference  Material:  AFI  33-114,  Air  Force  Weapon  Systems  Software  Management  Guidebook,  MIL- 
HDBK-61 A 

Other  Considerations:  Examples  of  work  products  that  may  be  a  part  of  a  configuration  item  include 
process  descriptions,  requirements,  design  details,  verification  methods  and  procedures,  source  code  and 
tools.  Other  documents,  such  as  interface  definitions  and  test  results,  may  also  be  included. 


CMG2P2  Establish  and  maintain  configuration  and  change  management 
systems. 

Description:  The  configuration  and  change  management  system  enables  the  disciplined  management  and 
control  of  changes  to  the  product  and  related  work  products. 

Typical  Work  Products: 
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□  Configuration  management  system  with  controlled  work  products 

□  Configuration  management  system  access  control  procedure 

□  Change  request  database 

Reference  Material:  AFI  33-114,  MIL-HDBK-61A,  Air  Force  Weapon  Systems  Software  Management 
Guidebook 


Other  Considerations:  The  configuration  management  system  includes  the  storage  media,  the 

procedures,  and  the  tools  for  accessing  the  configuration  system.  It  also  includes  the  storage  media,  the 
procedures,  and  the  tools  for  recording  and  accessing  change  requests.  The  system  should  be  able  to 
retrieve  documentation  of  any  baseline  and  status  of  all  CM  actions 


CMG2P3  Create  or  release  technical  baselines. 


Description:  Ensure  that  all  baselines  established  are  created  from  configuration  items  described  within 
the  configuration  management  system,  documented,  authorized  and  released  in  accordance  with  CM 
plans. 

Typical  Work  Products: 


□  Description  of  baselines 


Reference  Material:  AFI  33-114,  MIL-HDBK-61  A,  Air  Force  Weapon  Systems  Software  Management 
Guidebook 

Other  Considerations:  One  common  set  of  baselines  includes  the  product-level  requirements,  product- 
element-level  design  requirements  and  the  product  definition  at  the  end  of  development/beginning  of 
production.  These  are  typically  referred  to  as  the  functional,  allocated,  and  product  baselines.  The 
baselines  correlates  with  the  system  work  breakdown  structure  (WBS)  and  refer  to  both  functional  and 
physical/product  baselines. 


CMG2P4  Track  and  control  changes. 


Description:  Track  the  status  of  change  requests  to  closure.  Determine  the  impact  that  the  change  will 
have  on  the  product,  related  work  products,  use  of  the  product,  logistics  support,  schedule  and  cost. 
Maintain  control  over  the  configuration  of  the  work  product  baseline.  This  control  includes  tracking  the 
configuration  of  each  of  the  configuration  items,  approving  a  new  configuration  if  necessary,  and  updating 
the  baseline.  Maintain  the  integrity  and  traceability  of  the  configuration  throughout  the  product  life  cycle. 

Typical  Work  Products: 


□  Change  requests 

□  Revision  history 

□  Baseline  archives 


Reference  Material:  Air  Force  Weapon  Systems  Software  Management  Guidebook 


Other  Considerations:  Change  requests  address  not  only  new  or  changed  requirements  (e.g.,  due  to 
obsolescence,  and  technology  refreshment),  but  also  failures  and  defects  in  the  work  products. 
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CMG3  Integrity  of  baselines  is  established  and  maintained 

CMG3P1  Establish  and  maintain  records  describing  configuration  items 

Description:  Ensure  that  configuration  information  permits  forward  and  backward  traceability  to  other 
baselined  configuration  states. 

Typical  Work  Products: 

□  Revision  history  of  CIs 

□  Change  log 

□  Change  request  records 

□  Status  of  CIs 

□  Differences  between  baselines 

Reference  Material:  AFI  33-114,  Air  Force  Weapon  Systems  Software  Management  Guidebook,  MIL- 
HDBK-61 A 

Other  Considerations:  Records  include  revision  history  of  configuration  items,  change  log,  copy  of 
change  requests,  status  of  configuration  items,  and  differences  between  baselines. 

CMG3P2  Perform  configuration  audits  to  maintain  integrity  of  the  configuration 
baselines 


Description:  Audit  configuration  management  activities  and  processes  to  confirm  that  the  resulting 
baselines  and  documentation  are  accurate,  and  record  the  audit  results  as  appropriate. 

Typical  Work  Products: 

□  Configuration  audit  results 

□  Action  Items  to  address  discrepancies 


Reference  Material:  Air  Force  Weapon  Systems  Software  Management  Guidebook,  MIL-HDBK-61A 

Other  Considerations:  Functional  Configuration  Audits  are  used  to  verify  that  the  configuration  item  has 
achieved  the  performance  requirements  as  specified  in  its  functional  configuration  identification  documents. 
Physical  Configuration  Audits  are  the  formal  examination  of  as-built  or  as-coded  configurations  to  ensure 
that  products  produced  and  processes  used  match  the  approved  baseline  documentation.  Approach 
audits  as  a  series  of  rolling  reviews  in  which  items  are  progressively  audited. 


8.2  Decision  Analysis  (DA) 

The  purpose  of  Decision  Analysis  is  to  analyze  possible  decisions  using  a  formal  process 
that  evaluates  identified  alternatives  against  established  criteria. 

A  repeatable  criteria-based  decision-making  process  is  especially  important,  both  while 
making  the  critical  decisions  that  define  and  guide  the  acquisition  process  itself  and  later 
when  critical  decisions  are  made  with  the  selected  suppliers.  The  establishment  of  a 
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formal  process  for  decision-making  provides  the  acquisition  project  with  documentation  of 
the  decision  rationale.  Such  documentation  allows  the  criteria  for  critical  decisions  to  be 
revisited  when  changes  that  impact  project  requirements  or  other  critical  project 
parameters  change. 

The  Decision  Analysis  process  area  can  best  be  described  by  the  following  goals  and 
practices. 

DAG1  Base  decisions  on  an  evaluation  of  alternatives  using  established  criteria 

DAG1P1  Establish  and  maintain  guidelines  to  determine  which  issues  are 
subject  to  a  formal  evaluation  process 


Description:  Define  thresholds  and  authorities  for  controlling  resolution  of  issues  subject  to  a  formal 
evaluation  process  in  the  guidelines. 

Typical  Work  Products: 


□  Guidelines 


Reference  Material:  AFI  63-1201 

Other  Considerations:  Not  every  decision  is  significant  enough  to  require  a  formal  evaluation  process. 
The  choice  between  the  trivial  and  the  truly  important  may  be  unclear  without  explicit  guidance.  The 
significance  of  a  particular  decision  is  dependent  on  the  project  and  circumstances  and  is  determined  by 
the  established  guidelines.  Ensure  relevant  technical  issues  are  brought  to  the  designated  SE  technical 
authority  as  part  of  the  acquisition  decision  process. 


DAG1P2  Establish  and  maintain  the  criteria  for  evaluating  alternatives,  the 
relative  ranking  of  these  criteria,  and  select  the  evaluation  methods 


Description:  Consider  solution  costs,  risks,  technology  limitations,  and  impact  on  established  baselines 
and/or  operational  functions  in  the  criteria.  Include  rationale,  range,  and  scale  of  values,  and  coordinate 
with  appropriate  stakeholders. 

Typical  Work  Products: 


□  Evaluation  criteria  and  methods  (e.g.  Decision  analysis  record,  acquisition  strategy,  etc.) 


Reference  Material:  AFI  63-101 

Other  Considerations:  Regular  use  of  decision-making  criteria,  even  for  less  significant  decisions,  can 
be  extremely  helpful  in  creating  a  practice  for  disciplined  decision  making.  Practiced  evaluators  have 
demonstrated  that  defined  criteria  and  weighting  can  be  significant  contributors  to  the  speed  and 
consensus  level  of  a  decision.  Decision-making  criteria  may  apply  at  two  distinct  levels,  i.e. ,  at  the  project 
level  where  criteria  may  be  due  to  project  priorities,  and  at  the  issue  level  where  different  criteria  may  be 
appropriate  for  each  specific  issue.  The  decision  making  method  that  best  focuses  on  the  issue(s)  should 
be  selected  giving  consideration  to  impact  on  cost,  schedule,  performance,  and  risk.  Two  or  more  methods 
may  be  indicated  if  the  consequences  of  a  poor  decision  are  high.  The  evaluator  may  select  from  methods 
including  Cost  Study/Cost  Model,  Simulation/Prototype,  Manufacturing  Study, 
Engineering/Benchmark/Trade  Study,  Business  Opportunity  Study,  Extrapolation  based  on  experience  or 
study,  User  Review  and  Comment,  Testing,  Data  Comparison,  Feature  Comparison,  Decision  Tool  (e.g., 
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tool  for  Monte  Carlo  simulation  for  distribution  of  uncertainty  in  choices),  Modeling  Tool,  and  others. 
Stakeholders  and  project  management  should  concur  with  selected  methods. 


DAG1P3  Identify  alternative  solutions  to  address  issues 


Description:  Document  the  alternative  solutions  and  include  rationale  for  including  or  rejecting  any 
alternatives  for  consideration. 

Typical  Work  Products: 


□  Analysis  of  Alternatives  (AoA) 


Reference  Material: 

Other  Considerations:  A  wider  range  of  alternatives  can  surface  by  literature  search  on  the  applicable 
issue  and  by  soliciting  as  many  stakeholders  as  practical  for  input.  Input  from  stakeholders  with  diverse 
skills  and  backgrounds  can  help  teams  identify  and  address  assumptions,  constraints,  and  biases. 
Brainstorming  sessions,  with  support  from  the  stakeholders,  may  stimulate  innovative  alternatives  through 
rapid  interaction  and  feedback.  Additionally  to  stakeholders,  human  system  integration  experts  can  review 
alternatives  for  usability,  safety,  survivability  and  with  respect  to  other  human  requirements  and 
considerations.  HSI  reviews  and  recommendations  during  AOA  help  handle  risk  and  reduce  life  cycle 
costs. 


DAG1P4  Evaluate  alternative  solutions  using  the  established  criteria  and 
methods 


Description:  Document  scoring  and  conclusion(s),  using  selected  evaluation  methods  and  established 
criteria.  Document  unknowns  or  assumptions  made  about  the  solution,  including  observations  that  support 
or  disprove  those  assumptions.  Identify,  evaluate,  and  document  new  alternatives  along  with  rationale  in 
cases  where  all  current  alternatives  prove  unsatisfactory.  Identify  changes  to  evaluation  criteria.  Use 
iterative  analysis  as  necessary. 

Typical  Work  Products: 


□  Meeting  minutes 

Reference  Material: 

Other  Considerations: 

DAG1 P5  Select  the  solution(s)  from  the  alternatives  and  document  decisions 
based  on  the  evaluation 


Description:  Document  the  recommended  solution(s),  data,  associated  risks,  and  rationale  for  selection. 
Document  and  present  any  dissenting  opinions.  Review  results  of  evaluation  with  stakeholders  and  project 
management  for  concurrence.  Document  the  final  decision  with  associated  rationale  and  risks. 

Typical  Work  Products: 


□  Decision  analysis  record 


Reference  Material: 
Other  Considerations: 
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8.3  Design  (D) 

The  purpose  of  Design  is  to  conceive  and  proof  an  integrated  solution  that  satisfies 
product  requirements.  Solutions,  designs,  and  implementations  encompass  products, 
product  components,  and  product-related  life-cycle  processes  either  singly  or  in 
combinations  as  appropriate.  Beyond  the  level  of  the  product  design  where  responsibility 
has  been  assigned  to  suppliers,  it  is  the  acquirer’s  role  to  oversee  the  adequacy  of  the 
supplier’s  process  for  requirements  allocation  and  product-component  design. 

The  Design  process  area  focuses  on  product  design,  initial  implementation  and 
integration.  As  each  level  of  the  product  is  defined,  there  is  an  iterative  process  of 
allocation,  high-level  design,  and  requirements  definition  (for  the  next-lower  level). 

Product  design  consists  of  two  broad  phases  that  may  overlap  in  execution:  preliminary 
and  detailed  design.  Preliminary  design  establishes  product  capabilities  and  the  product 
architecture,  including  product  partitions,  product-component  identifications,  product 
states  and  modes,  major  inter-component  interfaces,  and  external  product  interfaces. 

Detailed  design  fully  defines  the  structure  and  capabilities  of  the  product  components. 
During  detailed  design,  the  product  architecture  details  are  finalized,  product  components 
and  interfaces  are  completely  defined  (detailed  in  the  context  of  containing  all  the 
information  needed  to  manufacture,  code,  or  otherwise  implement  the  design  as  a 
product  or  product  component). 

Product  integration  is  achieved  through  progressive  assembly  of  product  components,  in 
one  stage  or  in  incremental  stages,  according  to  a  defined  integration  sequence  and 
procedures.  A  critical  aspect  of  product  integration  is  the  management  of  interfaces  to  the 
products  and  between  product  components  to  ensure  compatibility  among  the  interfaces. 
Attention  should  be  paid  to  interface  management  throughout  the  project. 

Product  integration  can  be  conducted  incrementally,  using  an  iterative  process  of 
assembling  product  components,  evaluating  them,  and  then  assembling  larger  collections 
of  components.  This  process  may  begin  with  analysis  and  simulations  (e.g.  virtual 
prototypes  and  rapid  prototypes).  In  a  succession  of  builds,  the  simulated  product  is 
constructed,  evaluated,  improved,  and  reconstructed  based  upon  knowledge  gained  in 
the  evaluation  process. 

The  Design  process  area  can  best  be  described  by  the  following  goals  and  practices. 

DG1  The  design  is  based  upon  a  documented  architecture,  traceable  to 
requirements,  and  optimized  for  the  set  of  requirements  and  constraints 

DG1P1  Establish  and  maintain  the  architectural  design  baseline 


Description:  Develop  logical  and  physical  views  of  the  product  that  model  the  functionality  to  be 
performed  along  with  the  required  component  relationships.  Design  considerations  should  include  ease  of 
change,  technology  transparency  and  mitigating  the  risk  of  component  obsolescence.  Consider  the  use  of 
simulation  to  explore  alternative  models,  derive  other  requirements  and  better  communicate  needed 
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product  characteristics.  Iteratively  as  the  layers  of  the  product  architecture  are  defined  the  product's 
required  functionality  should  be  allocated  to  the  components.  Periodically  conduct  architecture  evaluations 
to  confirm  the  continued  suitability  of  the  product  architecture.  Review  the  computing  system  architecture 
to  ensure  robustness,  fault  tolerance,  reserve  capacity  and  growth  are  addressed. 

Typical  Work  Products: 


□  DoDAF  views 

□  Interface  descriptions 

□  Top  level  engineering  drawings 

□  Computer  System  &  Software  architectures 

Reference  Material: 

Other  Considerations: 


DG1P2  Establish  and  maintain  interface  designs 


Description:  Define  the  logical,  physical,  and  human  interfaces  for  the  product  and  between  product- 
components.  Managing  product  and  product-component  interfaces  must  start  very  early  in  the  development 
of  the  product.  The  definition  and  control  of  interfaces  affects  requirements  allocation,  product  design, 
manufacturing,  integration  and  verification.  Make  use  of  acceptable  standards.  Consider  interface 
requirements  and  product  partitioning  to  facilitate:  incorporation  of  security  technologies,  design  robustness 
and  replacement  of  obsolescent  parts  (e.g.,  military  grade  global  positioning  system  receivers). 

Typical  Work  Products: 


□  Interface  design  documents 

□  ICWG  charter  and  minutes 


Reference  Material: 
Other  Considerations: 


DG1P3  Establish  and  maintain  design  artifacts  that  describe  the  conditions, 
functions,  operating  modes,  and  operating  states  specific  to  the  components  of 
the  architecture 


Description:  Document  the  interaction  of  the  product-components  with  the  environment,  other  systems, 
and  end  users  including  modes  and  operating  states  for  a  particular  component  or  set  of  components. 
Select  among  alternatives  and  refine  the  architecture  that,  when  implemented,  will  satisfy  the  intended  use 
of  the  product.  Use  modeling,  analysis,  and  verification  techniques  to  the  maximum  extent  feasible. 

Typical  Work  Products: 


□  Operational  use  cases 

□  Sequence  and  collaboration  diagrams 

□  State  charts  or  diagrams 
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Reference  Material: 
Other  Considerations: 


DG1P4  Develop  potential  product-component  solutions,  alternatives,  and 
selection  criteria 


Description:  Base  alternatives  on  potential  product  architectures.  Explore  design  space  of  feasible 
solutions.  Use  robust  engineering  processes  to  ensure  a  quality  and  flexible  product.  Identify  the  key 
factors  and  selection  criteria  that  provide  a  basis  for  the  solution  (e.g.,  design  for  manufacturing, 
supportability,  and  reliability).  Identify  criteria  that  provide  clear  discrimination  and  an  indication  of  success 
in  arriving  at  a  balanced  solution  across  the  life  of  the  product.  Include  measures  of  cost,  schedule, 
performance,  supportability,  and  risk. 

Typical  Work  Products: 


□  Technical  review(s)  entry  &  exit  criteria 

□  Allocated  baseline 

□  Selection  criteria 


Reference  Material:  IEEE/EIA  12207,  MIL-HDBK-514 

Other  Considerations:  Modeling,  simulation  and  analysis  techniques  can  be  used  to  identify  and  minimize 
product  design  failure  modes  during  the  initial  design  phases. 

DG1 P5  Analyze  and  select  product-component  solutions  that  best  satisfy  the 
established  criteria 


Description:  Develop  a  modeling  and  simulation  activity  to  evaluate  potential  solutions  in  a  simulated 
operational  context,  wherever  feasible.  As  selections  are  made,  the  design  space  may  be  constricted  and 
other  alternatives  examined  until  the  most  promising  (i.e. ,  optimal)  solution  that  meets  requirements  and 
criteria  is  identified.  Document  the  description  of  the  alternative  solutions  and  the  rationale  for  selection. 

Typical  Work  Products: 


□  KPPs 

□  Interfaces 

□  M&S  results 

□  T rade  studies  w/  AoAs 

□  Critical  Operational  Issues  (COIs) 

□  TPMs 

□  Reliability  &  Maintainability  Analysis 

□  Usability  Analysis 
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Reference  Material: 

Other  Considerations: 

DG2  Develop  and  document  a  detailed  design  and  implementation  strategy 

DG2P1  Establish  initial  product-component  designs  and  development 
strategies 


Description:  Define  the  structure  and  capabilities  of  the  product-components.  Use  lower  level 
requirements  generated  from  the  selected  alternatives  to  develop  the  detailed  component  design.  Detail 
the  logical  and  physical  interfaces  between  product-components.  Develop  the  key  product/design 
characteristics  including:  anti-tamper,  materials,  fabrication,  logistics  support,  and  manufacturing 
requirements  and  processes.  Identify  constraints  on  implementation.  Consider  technology  maturity  and 
manufacturing  readiness  as  potential  areas  for  risk  mitigation  and/or  product  evolution. 

Typical  Work  Products: 


□  Development  hardware  and  software  specifications 

□  WBS  (Work  Breakdown  Structure) 

□  TRA  (Technology  Readiness  Assessment) 

□  MRA  (Manufacturing  Readiness  Assessment) 

□  Bill  of  materials 

□  ILSP  (Integrated  Logistics  Support  Plan) 

Reference  Material:  MIL-STD-881,  MIL-HDBK-881 ,  AFI  63-1201 
Other  Considerations: 


DG2P2  Evaluate  whether  the  product-components  should  be  developed, 
purchased,  or  reused  based  on  established  criteria 


Description:  Determine  what  products  or  product-components  will  be  acquired  -  this  is  frequently  referred 
to  as  a  make-or-buy  analysis.  It  is  based  on  an  analysis  of  the  needs  of  the  project.  Perform  industrial  base 
analysis  for  critical  products  and  product-components.  Analyze  cost,  schedule,  and  performance  risks 
when  evaluating  commercial  off-the-shelf  (COTS)  product-components 

Typical  Work  Products: 


□  Make-or-buy  analysis 

□  Industrial  base  analysis  (to  include  diminishing  manufacturing  sources) 


Reference  Material:  AFI  63-101,  AFI  21-118,  SD22,  DI-SESS-81656,  GEB1 

Other  Considerations:  The  make-or-buy  analysis  begins  early  in  the  project  during  the  first  iteration  of 
design,  continues  during  the  design  process,  and  is  completed  with  the  decision  to  develop,  acquire,  or 
reuse  the  product. 
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DG2P3  Establish  detailed  designs  for  the  product-component. 

Description:  Develop  the  key  product/design  characteristics  and  associated  manufacturing  processes 
including  materials  (bills  of  material  and  material  characteristics),  fabrication  and  manufacturing 
requirements  (for  both  the  original  equipment  manufacturer  and  new  supplier,  and  field  product  support. 

Typical  Work  Products: 

□  PHS&T  documentation 

□  TDP  (Tech  Data  Package) 

□  CDR  entry  and  exit  requirements,  and  action  item  closure 

□  Final  bill  of  materials 

□  Manufacturing  plan 


Reference  Material:  MIL-STD-2073,  AFI  63-501,  MIL-STD-1528A,  AFI  63-1201 

Other  Considerations:  The  detailed  design  results  in  the  specification  of  each  of  the  components  at  all 
levels  of  the  hierarchy.  It  addresses  each  of  the  key  factors  that  provided  a  basis  for  the  selection  of  the 
solution  (e.g.  design  for  manufacturing,  supportability,  producibility,  reliability,  safety,  packaging  and 
handling,  etc).  A  review  (traditionally  a  CDR)  process  is  used  during  design  to  include  both  hardware  and 
software  design  to  demonstrate  that  the  allocation  of  requirements  achieve  an  effective  system.  This 
activity  includes  the  allocation,  refinement  and  production  of  the  product-components,  which  can  be 
accomplished  using  the  planned  manufacturing  tools,  processes,  and  personnel.  It  also  includes  the 
coordination  between  the  various  product-component  development  efforts  to  refine  internal  interface 
specifications. 

DG2P4  Establish  and  maintain  a  technical  data  package 

Description:  Develop  a  comprehensive  technical  data  package  for  the  product  or  product-component  to 
support  development  and  sustainment. 

Typical  Work  Products: 


□  Installation  Plan 

□  User  Manual 

□  TO  (Technical  Orders) 

□  Training  Manual 

□  TCTO  (Time  Compliance  Technical  Orders) 

□  Digital  System  Models 

□  TDP  (Technical  Data  Package) 

□  Software  Requirements,  Design,  and  Verification  Documentation 

Reference  Material: 

Other  Considerations: 

The  technical  data  package  usually  includes  the  following: 

•  Component  design  and  life-cycle  process  descriptions, 
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•  Key  product/design  characteristics  including  functional  responsibilities,  logical  and  physical  interface 
requirements, 

•  Rationale  for  decisions  and  characteristics  (requirements,  requirement  allocations,  and  design  choices), 

•  Materials,  fabrication  and  manufacturing  requirements  (for  both  the  original  equipment  manufacturer  and 
field  support), 

•  The  verification  criteria  used  to  ensure  that  requirements  have  been  achieved  and 

•  End  user  documentation  that  will  be  used  to  install,  operate,  and  maintain  the  product. 


The  technical  data  package  may  also  contain  conditions  of  use  (environments)  and  operating/usage 
scenarios,  modes  and  states  for  operations,  support,  training,  manufacturing,  disposal,  and  verifications  to 
be  used  throughout  the  life  of  the  product. 


DG3  Assemble  the  design/development  prototype(s)  in  accordance  with  the 
detailed  design  and  integration  strategy. 

DG3P1  Establish  and  maintain  the  product  integration  approach 


Description:  Define  and  document  assembly  procedures  and  sequences  that  minimize  product  integration 
risks.  Identify  any  constraints  on  the  design  arising  from  the  integration  strategy.  Consider  requirements  for 
test  equipment,  test  software,  or  other  integration  items  such  as  fixtures.  Periodically  review  the  product 
integration  sequence  and  revise  as  needed. 

Typical  Work  Products: 


□  Assembly  procedures  and  sequences 

□  Interface  drawings  and  documents 

□  Integration  and  test  documents 

□  TCTO  (Time  Compliance  Technical  Orders) 

□  System  Integration  Plan 

□  Digital  System  Models 


Reference  Material:  AFI  99-103,  AFI  63-107,  T.O.  00-5-1 
Other  Considerations: 


DG3P2  Establish  and  maintain  procedures  and  criteria  for  integration  of  the 
product-components 


Description:  Procedures  for  the  integration  of  the  product-components  can  include  such  things  as  details 
of  the  expected  tests  and  other  evaluations  to  be  carried  out  at  each  stage.  Criteria  can  indicate  the 
readiness  of  a  product-component  for  integration  or  its  acceptability.  Assure  component  compatibility  with 
the  integration  environment.  Provide  sufficient  data  to  support  certifications. 

Typical  Work  Products: 


□  Product  and  product-component  certifications 

□  Integration  and  test  documents 
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Reference  Material:  AFI  99-103,  AFI  63-107 
Other  Considerations: 


DG3P3  Manage  internal  and  external  interface  definitions,  designs,  and 
changes  for  products  and  product-components 


Description:  Periodically  review  all  interface  descriptions  for  coverage  and  completeness.  Effective 
management  of  product-component  interface  requirements,  specifications,  and  designs  helps  ensure  that 
implemented  interfaces  will  support  integration. 

Typical  Work  Products: 

□  Assembly  procedures  and  sequences 

□  Interface  drawings  and  documents 

□  Integration  and  test  documents 

□  TCTO  (Time  Compliance  Technical  Orders) 

□  Digital  System  Models 


Reference  Material:  AFI  99-103,  AFI  63-107 

Other  Considerations:  Many  product  integration  problems  arise  from  unknown  or  uncontrolled  aspects 
of  both  internal  and  external  interfaces.  Participation  in  all  relevant  interface  control  working  groups 
(ICWGs)  helps  ensure  that  implemented  interfaces  will  support  integration. 


DG3P4  Conduct,  prior  to  assembly  product-component  verification 


Description:  Ensure  that  each  product-component  required  to  assemble  the  product  has  been  properly 
identified,  functions  according  to  its  description,  and  that  the  product-component  interfaces  comply  with  the 
applicable  interface  definition.  Develop  and  document  key  assembly  and  tightly  coupled  test  techniques 
during  first  article  production  for  use  in  subsequent  full-rate  production. 

Typical  Work  Products: 

Reference  Material: 

Other  Considerations: 


DG3P5  Assemble  product-components  according  to  the  product  integration 
sequence  and  established  procedures 


Description:  Acceptance  or  readiness  to  test  criteria  defined  during  design  and  integration  are  satisfied. 
This  assessment  can  also  be  performed  as  appropriate  for  different  stages  of  assembly  of  components  as 
identified  in  the  integration  sequence  and  available  procedures. 

Typical  Work  Products: 

Reference  Material: 
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8.4  Manufacturing  (M) 

The  purpose  of  the  Manufacturing  process  is  to  prepare  for  and  produce  the  required 
product. 

The  Manufacturing  process  area  includes:  application  of  industrial  base  and 
manufacturing  process  expertise  and  information  to  Requirements  and  Design 
processes;  planning  for  and  managing  the  manufacturing  process  maturation  efforts 
needed  for  successful  transition  from  product  development  to  rate  production;  and  then 
stabilizing  a  sustained  rate  production  while  assuring  affordable  quality  products.  Clear 
manufacturing  readiness  criteria  should  exist  for  each  phase  of  the  project  and  be  agreed 
to  by  relevant  stakeholders.  Manufacturing  readiness  assessments  (MRA)  should  be 
conducted  to  confirm  manufacturing  readiness  at  key  points  in  the  project.  Manufacturing 
transition  plans  are  established  to  address  the  manufacturing  readiness  criteria  and 
executed  to  ensure  maturation  of  manufacturing  capability.  The  residuals  of 
manufacturing  (e.g.  facilities,  processes,  tooling  and  test  equipment)  should  be  integrated 
into  the  support  infrastructure  required  for  the  remainder  of  the  product  life  cycle. 

The  Manufacturing  process  area  can  best  be  described  by  the  following  goals  and 
practices. 

MG1  Prepare  for  manufacturing 

MG1  PI  Establish  and  maintain  strategy  and  plans  for  manufacturing 


Description:  Describe  the  anticipated  approach  for  producing  the  end-item,  including  purchasing  material, 
parts,  and  subassemblies  and  in-house  fabrication,  assembly,  and  test.  Describe  the  resources  needed 
(tooling,  test  equipment,  facilities,  manpower,  and  special  skills)  and  the  plans  for  acquiring  them.  Describe 
the  supplier  management  procedures,  the  major/critical  suppliers,  the  Quality  Management  System,  and 
the  controls  established  to  ensure  conforming  products.  Describe  the  metrics  and  predictive  indicators  that 
will  be  used  to  assess  manufacturing  cost,  schedule,  and  quality  performance,  including  the  performance  of 
key  suppliers.  Describe  all  production-related  activities  that  must  be  accomplished  to  ensure  a  smooth 
transition  from  development  to  rate  production.  Describe  an  integrated  strategy  for  design  and 
manufacturing  engineering,  capital  investment  and  personnel  recruiting  and  retention.  Define  strategies  for 
control  and  flow  down  of  policies,  process  and  practices  to  sub-tier  suppliers. 

Typical  Work  Products: 


□  Manufacturing  plan 


Reference  Material:  AFI  63-501,  DoDI  5000.02,  AFI  63-1201 

Other  Considerations:  The  manufacturing  strategy  should  be  based  on  an  assessment  of  current 
manufacturing  capabilities  and  a  realistic  plan  for  development  of  any  required  new  capabilities.  The 
manufacturing  plan  defines  the  approach  for  producing  a  product  configuration  including  all  actions  that  are 
required  to  produce,  test  and  deliver  an  acceptable  product  on  schedule  and  at  minimum  cost.  Procedures 
for  the  integration  of  the  product  components  should  include  acceptability  criteria  and  details  of  the 
verification  tests  to  be  carried  out  at  each  stage.  The  manufacturing  plan  should  address  product  delivery 
schedules,  make-or-buy  decisions  and  manufacturing  resource  requirements  (e.g.,  facilities,  tooling,  capital 
equipment,  personnel).  The  materials,  fabrication  flow,  time  in  process,  tools,  test  equipment,  plant 
facilities,  and  personnel  skills  should  be  described  and  integrated  into  a  complete  sequence  and  schedule 
of  events.  It  is  essential  that  the  suppliers  be  considered  in  the  planning. 
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MG1P2  Perform  concurrent  design  and  manufacturing  engineering 


Description:  Develop  and  obtain  approval  on  the  factory  verification  plan  and  realistic  production-rate 
analysis  before  the  design  configuration  is  frozen.  Conduct  producibility  studies  and  ensure  they  include 
either  an  evaluation  of  the  capabilities  of  the  manufacturing  processes  to  meet  design  requirements  or 
approaches  to  improve  the  product’s  ease  of  manufacturing.  Manufacturing  engineers  sign-off  on 
engineering  drawings  as  part  of  the  design  approval  process  to  ensure  that  the  product  can  be 
manufactured  economically. 

Typical  Work  Products: 


□  Factory  verification  plan 

□  Producibility  studies 

□  Production  readiness  analysis 

□  Engineering  drawings 


Reference  Material:  AFI  63-501 

Other  Considerations:  Concurrent  engineering  is  a  key  factor  in  improving  the  quality,  development 
cycle,  production  cost,  and  delivery  time  of  products.  It  focuses  on  designing  the  manufacturing  processes, 
tooling  and  special  test  equipment  at  the  same  time  as  the  product  is  designed.  Design  for  Manufacturing 
(DFM)  is  an  activity  accomplished  through  the  collaboration  of  many  disciplines  including  design  and 
manufacturing  engineers  and  technicians.  Product  designers  must  understand  manufacturing  operations 
and  support  development  of  manufacturing  processes  and  procedures  to  ensure  that  the  product  can  be 
manufactured  efficiently  and  economically. 


MG1P3  Establish  and  maintain  manufacturing  technical  data 


Description:  Define  critical  manufacturing  processes  and  procedures,  their  key  characteristics  and  the 
metrics  by  which  their  effectiveness  will  be  measured.  Describe  the  production  test  requirements,  pass/fail 
criteria,  trend  analysis  methods,  and  archival  requirements. 

Typical  Work  Products: 


□  Critical  manufacturing  processes  and  procedures 

□  Drawings,  parts  lists,  process  flow  charts  including  inspection  operations,  production  documents 
(e.g.,  manufacturing  plans,  travelers,  routers,  work  orders,  process  cards,  etc.)  and  inspection 
documents 

□  A  list  of  tools  and  numerical  control  machine  programs  required  and  any  specific  instructions 
associated  with  their  use 


Reference  Material: 
Other  Considerations: 


MG2  Transition  from  development  to  repeatable  and  economical  production  at 
desired  rate 

MG2P1  Establish  and  maintain  plans  for  transition  to  production 
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Description:  Describe  an  integrated  strategy  for  design  and  manufacturing  engineering,  capital  investment 
and  personnel  recruiting  and  retention.  Include  provisions  to  ensure  that  manufacturing  personnel  influence 
the  design  process  to  ensure  the  final  configuration  is  capable  of  being  economically  produced  at  the 
desired  rates  and  with  adequate  quality.  Prepare  a  transition  plan  to  be  used  by  the  suppliers  during  the 
early  phases  of  development  to  support  tradeoff  decisions  which  can  have  a  significant  impact  on 
production. 

Typical  Work  Products: 


□  Transition  plan 


Reference  Material: 

Other  Considerations:  The  transition  to  production  plans  are  comprehensive  management  plans  that 
describe  all  production-related  activities  that  must  be  accomplished  during  design,  test,  and  low-rate  initial 
production  to  ensure  a  smooth  transition  from  development  to  rate  production. 


MG2P2  Qualify/proof  manufacturing  processes,  special  tools  and  test 
equipment 


□  Description:  Ensure  production  planning,  tool  design,  assembly  methods,  finishing  processes, 
personnel  training  and  special  processes  (e.g.  one-time  inspection  or  test  that  can  not  be  verified 
later  in  the  process)  are  performed  before  rate  production  at  the  integrator  and  major  supplier 
facilities.  Document  qualified  manufacturing  processes  and  place  them  under  configuration  control. 
Establish  proof  of  production  processes  where  the  resulting  output  (e.g.  single-shot  device,  bomb 
fuse,  or  satellite)  cannot  be  verified  by  subsequent  monitoring  or  measurement. 


Typical  Work  Products: 


□  Manufacturing  process  documentation 

□  Special  process  documentation 


Reference  Material: 

Other  Considerations:  As  early  as  possible  in  development,  proof  of  manufacturing  models  should  be 
used  to  establish  that  planned  production  processes  and  procedures  are  compatible  with  the  design. 
Development  hardware  should  be  produced  using  production  processes  and  procedures  to  the  maximum 
extent  possible. 


MG2P3  Ensure  readiness  for  manufacturing 


Description:  Ensure  readiness  for  manufacturing  by  performing  Manufacturing  Readiness  Assessments 
(MRA)  and  Production  Readiness  Reviews  (PRR).  Ensure  the  assessment  verifies  a  product  baseline 
(e.g.,  qualified  and  controlled  drawings  and  process  specifications)  that  can  be  produced  consistently  with 
the  facilities,  tooling,  processes,  and  personnel  available  within  the  production  budget/schedule.  Ensure  the 
assessment  identifies  manufacturing/quality  risks,  potential  metrics,  and  produces  a  manufacturing 
readiness  level  (MRL). 

Typical  Work  Products: 


□  MRA 

□  PRR  report  or  briefing 
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Reference  Material: 

Other  Considerations:  MRAs  can  determine  the  adequacy  of  the  manufacturing  processes,  the  Quality 
Management  System,  and  the  production  plans.  PRR  criteria  address  the  evaluation  of  design  maturity, 
critical  manufacturing  process  capabilities,  expected  variability,  critical  spares,  product  acceptance  criteria, 
tooling,  test  equipment,  personnel,  quality  controls  to  ensure  conforming  products,  supplier  readiness  and 
capabilities,  production  cost  estimates,  and  manufacturing  risks  throughout  the  supply  chain.  Production 
Readiness  Review  planning  is  completed  with  entry  and  exit  criteria  established. 


MG3  Manufacture  the  product  in  accordance  with  plans  and  specifications 

MG3P1  Ensure  that  production  at  desired  rates  is  conducted  according  to  the 
plan 

Description:  Ensure  the  manufacturing  processes  and  procedures  adhere  to  the  approved  plan(s)  to 
provide  a  uniform,  quality  product  with  consistent  performance.  Track  and  report  metrics  to  maintain 
insight  into  the  manufacturing  operations  (e.g.  actual  hours  per  ship  set  or  lot,  realization  or  efficiency, 
traveled  work,  cycle  time,  scrap/rework/repair,  cost  of  quality,  process  capabilities  (Cpk),  MRB  dispositions, 
quality  escapes,  and  first  pass  yields). 

Typical  Work  Products: 


□  Work  control  document 

□  Process  control  charts 

□  Statistical  analyses 


Reference  Material: 

Other  Considerations:  Take  advantage  of  Defense  Contract  Management  Agency  (DCMA)  processes 
and  personnel  as  much  as  possible. 


MG3P2  Establish  and  maintain  inventory  and  supplier  management/control 


Description:  Establish  a  supplier  management  program  to: 

•  Communicate  project  requirements  with  the  suppliers, 

•  Assure  that  manufacturing  readiness  levels  at  key  suppliers  support  project  needs, 

•  Assure  supplier  compliance  with  requirements, 

•  Address  risks  associated  with  transferring  work  between  facilities, 

•  Identify  and  manage  supplier  risks  and 

•  Continuously  assess  overall  health  of  the  supply-chain 

Perform  site  visits  to  assure  process  plans  are  documented  and  followed  to  ensure  that  the  chosen 
supplier(s)  can  perform.  Make  follow-up  visits  to  control  process  drift.  Conduct  conferences  tailored  to 
educating  each  supplier  on  their  contract  as  well  as  the  key  technical  elements  of  the  overall  project. 

Typical  Work  Products: 
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□  Supplier  management  plan 

□  Inventory  control  plan 

□  Risk  matrix 


Reference  Material: 

Other  Considerations:  Good  communication  among  the  acquirer  and  suppliers  is  a  key  ingredient  in 
effective  supply-chain  management.  The  project  team  should  consider  using  a  dedicated,  multi-discipline 
onsite  evaluation  team  (including  representatives  from  production,  quality,  material,  technical,  and 
configuration  management)  to  ensure  completeness  and  consistency  of  specifications,  procurement 
packages,  technical  interfaces,  and  flow  down  of  requirements  to  the  suppliers.  The  type  and  extent  of 
control  applied  to  the  supplier  and  the  purchased  product  should  be  dependent  upon  the  effect  of  the 
purchased  product  on  the  final  product. 


MG3P3  Complete  First  Article  Inspection  (FAI) 


Description:  Conduct  FAI  (inspection  and  verification)  on  a  representative  item  from  the  first  production 
run  of  a  new  part,  or  following  any  change  that  invalidates  the  previous  first  article  inspection  result. 
Subject  the  first  article  to  a  formal  examination  (e.g.,  Physical  Configuration  Audit  (PCA))  to  ensure  the  as- 
built  configuration  matches  the  draft  technical  data  package  and  also  validates  supporting  production 
processes. 

Typical  Work  Products: 


□  FAI  report 

□  PCA  minutes 

□  Action  items 


Reference  Material: 
Other  Considerations: 


MG4  Product  and  process  quality  is  assessed  and  improved 

MG4P1  Establish  and  maintain  piece  part  control  and  perform  manufacturing 
screening 

Description:  Flow  down  requirements  to  suppliers  and  require  a  formal  parts  control  program  during 
development.  Develop  testing  and  screening  programs  at  the  suppliers’  facilities,  and  receiving  inspection 
at  the  integrators’  facilities.  Monitor  the  supplier’s  performance  by  yield. 

Typical  Work  Products: 

□  Test  plans/procedures 

□  Parts  control  program  plan 


Reference  Material: 
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Other  Considerations:  A  key  element  of  parts  control  is  an  established  corporate  policy  which  ensures 
that  certain  steps  are  taken  to  control  part  quality  (both  electrical  and  mechanical),  independent  of 
government  contract  requirements.  Environmental  stress  screening  (ESS)  should  be  designed  during  the 
development  phase. 


MG4P2  Establish  and  maintain  a  quality  management  system 


Description:  Establish  and  maintain  manufacturing  processes.  Monitor  and  control  critical  processes  and 
product  variation.  Establish  mechanisms  for  feedback  of  field  product  performance.  Implement  a  root 
cause  analysis  and  corrective  action  system.  Apply  continuous  process  improvement  practices.  Establish 
processes  and  procedures  for  ensuring  implementation  of  these  activities  at  all  supplier  levels. 

Typical  Work  Products: 


□  Deficiency  reporting  system 

□  Quality  Assurance  Plan 


Reference  Material:  AFI  63-501 

Other  Considerations:  A  key  aspect  to  consider  is  how  process  changes  will  be  accepted,  authorized, 
documented,  and  implemented  to  establish  and  maintain  the  desired  quality. 


MG4P3  Establish  and  maintain  defect  control 


Description:  Design  products,  manufacturing  processes  and  tooling  to  minimize  the  potential  for  incorrect 
manufacturing  and  assembly  (i.e. ,  poka-yoke  or  fool  proofing).  Establish  deficiency  feedback,  root  cause 
analysis,  and  corrective  action  mechanisms.  Establish  a  corrective  action  team  to  ensure  adequate 
attention  to  the  causes  of  defects.  Develop  criteria,  processes  and  procedures  for  determination  and 
disposition  of  nonconforming  products  to  prevent  its  unintended  use  or  delivery. 

Typical  Work  Products: 


□  Deficiency  reporting  system 

□  Root  cause  and  corrective  action  analysis 


Reference  Material:  T.O.  00-35D-54 


Other  Considerations:  Management  commitment  to  a  culture  of  defect  “prevention”  is  the  essence  of 
defect  control.).  An  automated  defect  reporting/tracking  system  can  enhance  timely  analysis  of  problem 
areas  and  verification  of  effectiveness  of  action  taken  to  correct  problems.  TO  00-35D-54describes  the 
deficiency  reporting,  investigation,  and  resolution  process  for  USAF  products.  Process  Failure  Mode, 
Effects  and  Criticality  (PFMECA)  are  performed  to  identify  potential  failures  in  critical  and  safety  related 
manufacturing  processes  and  identify  actions  to  mitigate  those  potential  failures. 


8.5  Project  Planning  (PP) 

The  purpose  of  Project  Planning  is  to  establish  and  maintain  plans  that  define  project 
activities.  Project  Planning  leads  project  members  to  think  about  and  define  all  the  activities 
required  to  achieve  the  goals  of  the  project.  Typical  activities  include  defining  the  formal 
procedures  to  be  used,  such  as  the  creation  of  documents,  diagrams,  or  meetings  to  discuss 
the  important  issues,  the  objectives  to  be  met,  and  the  strategies  to  be  followed  and  an 
understanding  of  roles  and  responsibilities. 
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Planning  starts  by  aligning  the  technical  activities  with  the  acquisition  strategy  and  is 
followed  by  planning  technical  activities  in  ever-increasing  levels  of  detail.  The  resulting 
plans  should  be  reviewed  for  consistency  with  the  overall  acquisition  plans.  The 
acquirer’s  and  suppliers’  project  planning  processes  are  continuous,  and  the  plans  evolve 
to  meet  the  project’s  needs. 

This  area  relates  the  technical  objectives  for  the  acquisition,  the  constraints,  availability  of 
assets  and  technologies,  accommodation  of  end  user  considerations,  consideration  of 
risk,  and  technical  support  for  the  project  over  the  life  cycle. 

If  an  existing  product  is  to  be  replaced  as  part  of  the  effort,  the  acquirer  may  be  required 
to  consider  transition  from  operation  and  the  demilitarization/disposal  of  the  existing 
product  as  part  of  the  technical  planning  for  executing  the  new  product. 

The  Project  Planning  process  area  can  best  be  described  by  the  following  goals  and 
practices. 

PG1  Establish  and  maintain  estimates  of  project  planning  parameters 

PPG1P1  Define  the  project  life  cycle  phases,  milestones,  and  key  decision 
points 


Description:  Include  in  the  planning,  the  entire  known  life  cycle  from  user  needs  through  initial  and 
subsequent  upgrades.  Develop  entry  and  exit  criteria  for  each  project  milestone  or  key  decision  points. 

Typical  Work  Products: 

□  Integrated  Master  Plan 

□  Integrated  Master  Schedule 

□  Life  Cycle  Management  Plan  (LCMP) 


Reference  Material:  AFI  63-1201,  DoDI  5000.02, 

Other  Considerations:  Typical  life  cycle  choices  include  single-step  acquisition,  evolutionary 

incremental,  or  evolutionary  spiral.  Consider  the  full  collection  of  contracts  within  a  project  context  so  that 
an  integrated  approach  results,  rather  than  dealing  with  activities  individually.  This  supports  the  project 
planning  activities  on  the  occasions  when  some  elements  of  the  acquisition  or  life  cycle  may  be  out  of  the 
control  of  a  particular  acquisition  organization.  Consider  major  external  dependencies.  All  milestones  and 
tasks  are  linked  logically  to  provide  a  fully  integrated  schedule  demonstrating  a  critical  path  to  each  event 
as  well  as  towards  the  completion  of  the  project  itself. 


PPG1P2  Establish  a  work  breakdown  structure  (WBS)  to  organize  the  effort 


Description:  Define  tasks  in  the  WBS  for  the  work  of  the  project  to  include  activities  performed  by  the 
acquirer  as  well  as  the  suppliers  and  other  stakeholders  to  ensure  the  planning  effort  covers  the  full  scope. 

Typical  Work  Products: 
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□  WBS 

□  Work  packages 


Reference  Material:  MIL-STD-881 ,  MIL-HDBK-881 

Other  Considerations:  The  WBS  should  typically  be  down  to  the  lowest  level  of  work  managed. 


PPG1 P3  Establish  and  maintain  the  scope  of  the  work  products  and  tasks  that 
describe  the  project  cost  and  schedule. 


Description:  Define  the  attributes  (e.g.  weight,  size,  software  lines  of  code,  reliability,  security  and 
resource  requirements,  and  signature)  that  scope  each  product-component  and  task  that  are  of  concern  to 
the  project.  Estimates  of  the  attributes  of  the  work  products  and  tasks  are  then  used  to  bound  the  budget 
and  schedule.  Conduct  Integrated  Baseline  Reviews  (IBR). 

Typical  Work  Products: 


□  WBS 

□  Work  packages 

□  Cost  accounts 

□  Staffing  plans 

□  Software  build  plans 


Reference  Material: 

Other  Considerations:  Periodic  reviews  of  work  packages  and  activities  are  performed  against  allocated 
resources,  requirements  and  developer  plans. 


PPG1P4  Establish,  validate,  and  maintain  estimates  for  cost  and  schedule 


Description:  Develop  cost  and  schedule  estimates  of  the  project  work  products.  Require  independent 
review  (i.e.  sufficiency  review)  of  the  estimates  by  individuals  external  to  the  project  to  ensure  that  the 
project  estimation  is  validated. 

Typical  Work  Products: 


□  Work  packages 

□  Cost  accounts 


Reference  Material:  Air  Force  Weapon  Systems  Software  Management  Guidebook,  15  Aug  2008, 
Version  1.0,  Section  3.2,  Estimating  Software  Size,  Effort,  and  Schedule. 

Other  Considerations: 

For  Software: 

Software  development  and  integration  effort  (staff  hours),  cost,  and  schedule  are  estimated  conservatively; 
account  for  applicable  software-related  risks  and  uncertainties  (such  as  software  size  growth,  software 
reuse  erosion,  unstable/incomplete  requirements,  developer  capability,  etc.);  are  consistent  with  the 
proposed  software  development  approach  and  processes,  and  are  established  as  ranges  with  associated 
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probabilities,  (i.e.  10%,  50%,  and  90%  confidence).  The  software  estimates  have  been  verified  to  be 
consistent  with  the  program  budget  and  schedule. 

PPG2  Establish  and  maintain  integrated  plans 

PPG2P1  Assign  responsibility  for  acquisition  and  sustainment  management, 
support,  and  product  enhancement. 


Description:  Define  the  roles  and  responsibilities  of  the  user,  acquirer,  suppliers,  testers  and  enablers  for 
the  life-cycle  of  the  product.  Ensure  that  relevant  stakeholders  are  involved  early  in  the  acquisition  planning 
processes  by  explicitly  identifying  organizational  responsibility  for  support  and  enhancements.  Establish 
necessary  support  agreements  including  details  of  the  limitations  of  delegated  authority.  Use  the  USAF 
Mission  Area  Assignment  Process  (MAAP).  Identify  and  maintain  criteria  for  assigning  enhancements 
responsibility.  Use  the  DSOR  (Depot  Source  of  Repair)  process  to  define  the  source  of  repair  activities. 

Typical  Work  Products: 


□  Life  Cycle  Management  Plan 

□  Project  Management  Plan 

□  Mission  assignment  memos 

□  Source  of  Repair  Assignment  Process 


Reference  Material: 

Other  Considerations:  Planning  should  result  in  a  documented  management  plan. 


PPG2P2  Establish  and  maintain  engineering  plans  to  accomplish  project. 

Description:  Develop  the  technical  plans  to  execute  the  project.  Describe  the  approach  to  integrating 
systems  engineering  with  overall  project  management  efforts  (e.g.,  risk  management,  verification,  and 
logistics/supportability  objectives).  Describe  the  project’s  requirements  and  constraints,  processes  to  be 
employed,  approach  to  technical  staffing  and  organization,  baseline  management,  and  reviews.  Ensure 
project  risks  are  considered  in  planning  activities. 

Typical  Work  Products: 


□  Systems  Engineering  Plan 

□  HSI  Plan 

□  Software  Development  Plan 

□  Supportability  Plan 

□  System  Safety  Plan 

□  Test  and  Evaluation  Strategy  (TES)  and  Test  and  Evaluation  Master  Plan  (TEMP) 


Reference  Material:  AFI  63-1201,  Defense  Acquisition  Guide 

Other  Considerations: 
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PPG2P3  Plan  for  the  management  of  project  data. 


Description:  Consider  how  technical  data,  to  include  informal  data  as  well  as  formal  project  data  and 
plans,  will  be  shared  across  all  relevant  stakeholders.  Include  plans  for  managing  data  within  integrated 
teams  and  the  infrastructure  required  to  manage  data  between  the  suppliers,  operational  users,  and  other 
relevant  stakeholders.  Decide  which  project  technical  data  and  plans  require  version  control,  or  more 
stringent  configuration  control,  and  establish  mechanisms  to  ensure  project  data  are  controlled.  Plan  for 
how  technical  data,  test  results,  project  milestone  events,  etc,  will  be  evaluated  for  release  to  the  public  or 
outside  the  project. 

Typical  Work  Products: 


□  Data  Management  Plan 

□  Security  Classification  Management  Plan 


Reference  Material: 

Other  Considerations:  Consider  the  implications  of  controlling  access  to  classified  information  and 
sensitive  but  unclassified  information  (e.g.,  proprietary,  export  controlled,  source  selection  sensitive)  and 
other  access  controlled  data. 


PPG2P4  Plan  for  necessary  resources,  including  personnel  knowledge  and 
skills,  to  perform  the  project  tasks 


Description:  Plan  for  personnel  resources  (e.g.,  acquirer,  suppliers  or  other)  for  the  project  as  well  as 
tools,  infrastructure  and  services  required  during  the  life  of  the  project.  Include  consideration  for  integration 
and  test  facilities.  Include  orientation  and  training  for  project  team  and  stakeholders  in  acquisition  technical 
practices  and  the  domain  knowledge  required  to  execute  the  project. 

Typical  Work  Products: 


□  Staffing  plan 

□  POM  input 

□  SOW 

□  IMP 

□  Training  Plan 

□  Test  and  Evaluation  Strategy  (TES)  and  Test  and  Evaluation  Master  Plan  (TEMP) 


Reference  Material: 

Other  Considerations:  Knowledge  and  skill  requirements  can  derive  from  project  risk.  For  example,  if  the 
project  team  is  acquiring  a  software-intensive  product,  ensure  sufficient  expertise  in  systems  and  software 
engineering  exists  within  the  project  team.  Resource  planning  includes  implementation  of  Training 
Systems.  To  effectively  review  and  manage  software  acquisition  activities,  the  project  team  needs 
expertise  in  acquisition,  management,  and  software  development. 


PPG2P5  Plan  the  involvement  of  identified  stakeholders 
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Description:  Plan  for  involvement  of  stakeholders  from  other  projects  or  communities  to  ensure  the 
delivered  product  can  perform  as  required  in  its  intended  environment  when  it  has  to  interoperate  with  other 
products. 

Typical  Work  Products: 


□  Memorandum  of  Agreement 

□  Memorandum  of  Understanding 

□  Expectation  Management  Agreement 

□  Service  Level  Agreement 


Reference  Material: 

Other  Considerations:  Stakeholders  can  include  operational  users  and  project  participants  from  test  or 
other  support  communities  as  well  as  potential  suppliers. 


PPG2P6  Establish  and  maintain  the  technology  development  strategy 


Description:  Identify  potential  technology  constraints  and  develop  an  approach  for  overcoming  each 
constraint  using  appropriate  mitigation  approach  and/or  technology  insertion  at  the  appropriate  time  in  the 
life  cycle.  Provide  mechanisms  for  identifying  technology  alternatives.  Plan  activities  and  establish  criteria 
for  assessing,  validating,  and  transitioning  emerging  technologies. 

Typical  Work  Products: 


□  Technology  Development  Strategy  (TDS) 


Reference  Material: 

Other  Considerations:  Consider  plans  for  parallel  technology  development.  Inputs  to  the  planning 
process  include  Technology  Readiness  Assessments  and  Manufacturing  Readiness  Assessments. 


PPG2P7  Plan  for  product  Reliability,  Availability,  and  Maintainability  (RAM) 


Description:  Define  the  activities  (e.g.,  reliability  modeling,  failure  modes  effects  and  criticality  analyses, 
accelerated  life  testing,  reliability  demonstration  testing)  needed  to  achieve  and  demonstrate  the  product 
RAM  requirements. 

Typical  Work  Products: 


□  RAM  plans 


Reference  Material:  Acquisition  Sustainment  Tool  Kit 

Other  Considerations:  RAM  needs  to  address  consideration  of  fault  detection  and  isolation,  false 
alarms,  redundancy,  degraded  system  management,  modularity,  instrumentation  and  monitoring,  mitigation 
of  failure  modes,  maintenance  procedures  and  training,  and  manufacturing  and  quality  assurance  (as 
applicable).  RAM  is  representative  of  all  ilities”  that  may  exist  on  a  project. 


PPG2P8  Establish  and  maintain  an  Integrated  Master  Plan  and  Integrated 
Master  Schedule  (IMP/IMS) 
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Description:  Develop  an  event-driven  plan  that  describes  the  events,  significant  accomplishments,  and 
accomplishment  criteria  needed  to  complete  the  project  tasks  described  in  the  work  breakdown  structure. 
Develop  a  schedule  of  events  and  significant  accomplishments  against  a  timescale.  Develop  entry  and  exit 
criteria  for  each  project  milestone  or  key  decision  points. 

Typical  Work  Products: 


□  IMP 

□  IMS 

□  Entry/exit  criteria 


Reference  Material: 

Other  Considerations:  Consider  major  external  dependencies.  All  milestones  and  tasks  are  linked 
logically  to  provide  a  fully  integrated  schedule  demonstrating  a  critical  path  to  each  event  as  well  as 
towards  the  completion  of  the  program. 


PPG3  Establish  and  maintain  commitment  to  the  technical  plan 

PPG3P1  Review  all  plans  to  understand  commitments  and  ensure  the  technical 
plans  and  overall  plans  are  integrated  and  consistent 


Description:  Periodically  review  all  plans  to  ensure  they  are  clear,  and  integrated  and  consistent  with  one 
another.  Ensure  a  common  understanding  of  the  scope,  objectives,  roles  and  relationships,  authority, 
responsibility,  accountability,  and  control  for  all  project  technical  activities. 

Typical  Work  Products: 


□  All  project  plans 

Reference  Material: 

Other  Considerations:  The  project  may  have  a  hierarchy  of  plans  (e.g.,  Life  Cycle  Management  Plan, 
Risk  Management  Plan,  Systems  Engineering  Plan,  Test  and  Evaluation  Master  Plan)  in  addition  to 
stakeholder  plans  (e.g.,  Operational  Test,  Support,  Suppliers’  plans).  The  SEP  should  contain  references 
to  all  technical  plans  developed  for  the  project  (e.g.  Information  Support  Plan,  Program  Protection  Plan, 
Airworthiness  Plan  etc.).  Ensure  contractual  plans  implement  proposed  development  processes  and 
associated  products  (for  example,  ensure  the  IMP  and  IMS  include  appropriate  software  development 
processes  as  documented  in  the  Software  Development  Plan) 

PPG3P2  Reconcile  the  technical  plans  to  reflect  available  and  estimated 
resources 


Description:  De-scope  the  effort  to  match  available  resources  when  resources  (to  include  personnel, 
facilities,  stakeholders,  schedule,  and  funding)  are  inadequate  to  accomplish  the  technical  effort.  When  this 
is  not  feasible,  identify  and  handle  these  risks. 

Typical  Work  Products: 


□  Planning  documents 

□  Requirements  documents 
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□  Funding  documents 

□  IMP 

□  IMS 

□  Contract 


Reference  Material: 
Other  Considerations: 


PPG3P3  Obtain  commitment  from  relevant  stakeholders  responsible  for 
performing  and  supporting  execution 


Description:  Coordinate  and  obtain  approval  on  major  plans  from  relevant  stakeholders.  Identify  needed 
support  and  negotiate  commitments  during  plan  development.  Document  commitments  to  ensure  a 
consistent  mutual  understanding,  as  well  as  for  tracking  and  maintenance. 

Typical  Work  Products: 


□  Signed  project  plans 


Reference  Material: 

Other  Considerations:  Obtaining  commitment  involves  interaction  among  all  relevant  stakeholders 
(acquisition,  sustainment,  support,  test,  operations,  product  enhancement)  both  internal  and  external  to  the 
project.  The  individual  or  group  making  a  commitment  should  have  the  confidence  that  the  applicable 
work/support  can  be  performed  within  cost,  schedule,  and  performance  constraints.  Well-defined  interface 
specifications  may  form  the  basis  for  some  commitments. 


8.6  Requirements  (R) 

The  purpose  of  the  Requirements  process  area  is  to  develop  and  analyze  operational 
user,  product,  and  product-component  requirements,  to  assure  consistency  between 
those  requirements  and  the  project’s  technical  plans  and  work  products  and  to  manage 
requirements  evolution  through  the  life  cycle  of  the  product. 

The  requirements  process  has  several  contexts.  The  first  context  is  the  amalgamation 
and  coordination  of  the  stakeholder  requirements  (e.g.  operational  requirements)  into  a 
set  of  requirements  that  will  define  the  scope  and  direction  of  the  acquisition.  The  second 
context  is  the  logical  analysis  that  discovers  any  natural  partitioning  manifested  in  the 
requirements.  The  third  context  concerns  the  extension  of  the  customer  requirements 
and  additional  acquirer  requirements  derived  from  design  activities  that  occur  as  the 
product  matures  and  evolves  (e.g.,  product  characteristics,  architecture  requirements, 
component  design  requirements). 

There  is  a  continuous  iteration  developing  increasingly  detailed  derived  requirements  as 
the  multiple  layers  of  a  complex  product  are  defined.  For  example,  requirements  flow 
from  the  stakeholders  to  the  product,  segment,  sub  segment,  and  eventually  to  hardware 
or  software  component  levels.  The  responsibility  for  developing  requirements  down 
through  the  levels  is  generally  split  between  the  acquirer  and  the  suppliers.  The  acquirer 
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is  generally  responsible  for  the  higher  levels,  starting  with  operational  requirements,  and 
the  suppliers  are  generally  responsible  for  lower  levels.  The  division  of  responsibilities 
between  the  acquirer  and  suppliers  is  determined  for  each  project. 

The  acquirer  is  responsible  for  defining  and  baselining  the  requirements  levels  under  its 
control  and  also  monitoring  the  suppliers  definition  of  the  lower  level  requirements.  The 
acquirer  will  provide  direct  management  of  acquirer-controlled  requirements  and 
oversight  of  suppliers’  requirements  management.  Requirements  should  be  managed 
and  maintained  with  discipline  so  that  changes  are  not  executed  without  recognizing  the 
impact  to  the  project. 

The  Requirements  process  area  can  best  be  described  by  the  following  goals  and 
practices. 

RG1  Stakeholder  needs,  expectations,  constraints,  and  interface  requirements 
are  collected  and  translated  into  a  definition  of  needed  product 
capabilities/characteristics  for  all  phases  of  the  life  cycle. 

RG1P1  Elicit  stakeholder  needs,  expectations,  constraints,  and  interfaces 


Description:  Identify  all  stakeholders  with  interest  in  the  product.  Establish  mechanisms  for  eliciting, 
coordinating,  and  formalizing  operational  needs  and  constraints.  Include  non-functional  requirements  and 
other  attributes  such  as  life  cycle  cost,  need  date,  design  robustness,  producibility,  supportability,  re¬ 
usability,  HSI,  NEPA,  and  demilitarization/disposition  which  can  drive  the  definition  of  the  product.  High- 
level  interface  requirements  for  interoperability  in  any  system  of  systems  architecture  should  be 
considered.  Identify  and  assess  potential  requirements  or  constraints  that  may  evolve  from  treaty,  statutory, 
or  regulatory  considerations. 

Typical  Work  Products: 


□  Stakeholder  lists 

□  MOUs  &  MOAs 

□  Interface  requirements  specification  documents 

□  DoDAF  views  of  requirements  and/or  system  architecture 

□  CRRA  (Capabilities  Review  &  Risk  Assessment) 


Reference  Material:  CONOPS  IAW  CJCSI  3170.01,  Analysis  Handbook  (OAS/DR),  AFI  63-1201,  AFI 
10-601,  AFI  10-602,  CJCSI  6212.01 ,  AF  Policy  1 .2.1.3,  ,  DoDI  5000.02,  AFI  33-103 

Other  Considerations:  Requirements  constraints  may  include  technology  and  interface  requirements,  as 
well  as  non-technical  constraints,  e.g.,  policies,  standards,  legal,  budget,  and  schedule  constraints. 
Technology  constraints  include  time-critical  functions,  timing  dependencies,  fault  tolerance,  redundancy, 
and  graceful  degradation.  Human  interface  requirements  should  be  consistent  with  human  factors 
principles. 


RG1 P2  Establish  and  maintain  concepts  of  operations  and  support  that  define 
the  operational  capability  required 
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Description:  Document  the  interaction  of  the  product  with  the  environment,  other  systems,  and 
operational  users.  Obtain  user  concurrence  on  top  level  requirements  and  mission  profiles.  A  conceptual 
set  of  mission  profiles  (i.e. ,  use  cases)  fully  describing  the  product  boundaries,  intended  usage  concepts, 
modes  of  operation  and  environments  must  be  established,  maintained,  and  validated  by  the  appropriate 
subject  matter  experts  (e.g.,  HSI  community,  intelligence  community).  Key  capabilities  should  be  described 
in  specific  scenarios  with  explicit  measures  of  operational  effectiveness  and  suitability. 

Typical  Work  Products: 


□  Coordinated  and  Signed  CONOPS  (All  Parties) 

□  Use  cases 

□  Hierarchy  of  performance  parameters  (e.g.,  Key  Performance  Parameters  (KPPs),  Technical 
Performance  Measures  (TPMs)) 

□  Performance  Requirements  Summary  (Services) 


Reference  Material:  CONOPS  IAW  CJCSI  3170.01,  Analysis  Handbook  (OAS/DR),  AFI  63-1201,  AFI 
10-601,  AF  Policy  1.2.1. 3,  AFI  63-124 

Other  Considerations:  Use  cases/mission  profiles  should  consider  off-nominal  potential  operational 
concepts.  .  For  services  contracts  (i.e.,  repair,  overhaul),  brainstorm  with  stakeholders  of  all  dependent 
variables  (what,  when,  where,  who,  quantity,  quality  levels,  etc.)  to  ensure  that  all  unique  requirements 
have  been  considered.  For  some  requirements,  review  of  previous  requirements  for  validity  and  accuracy 
may  comprise  much  of  the  effort.  Determine,  prioritize,  and  activate  sustainment  actions  to  best  address 
present  and  life-cycle  system  operational  scenarios. 


RG1P3  Transform  stakeholder  needs,  expectations,  constraints,  and 
interfaces  into  a  prioritized  requirements  baseline 


Description:  Transform  stakeholder  needs,  expectations,  constraints,  and  interfaces  into  a  prioritized 
requirements  baseline  that  will  support  requirements  traceability  and  document  related  decisions  to 
manage  requirements  evolution  throughout  development.  Establish  technical  aspects  of  a  solicitation 
package  that  include  technical  requirements  of  the  acquisition  and  corresponding  proposal  evaluation 
criteria. 

Typical  Work  Products: 


□  System  Requirements  Document/Performance  Specification 

□  Performance  Work  Statement,  for  services 

□  ICD 

□  SOW,  SOO 

□  Requirements  list  Performance  Work  Statement  (for  services  contracts) 


Reference  Material:  AFI  63-124 

Other  Considerations:  Document  requirements  that  have  been  identified  through  concept  refinement  or 
previous  product  development.  Both  hardware  and  software  need  to  be  considered  in  the  specification  of 
functional,  performance,  and  interface  requirements. 
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RG1P4  Establish  and  maintain  a  requirements/decision  data  archive  to 
document  requirements  and  related  technical  decisions 


Description:  Stakeholder  needs,  expectations,  constraints,  and  interfaces  are  documented  in  a  database 
of  the  end-item  requirements  and  are  used  to  manage  overall  performance  expectations  throughout  the 
development,  enable  transfer  to  sustainment  activities,  and  support  requirements  traceability. 

Typical  Work  Products: 


□  Evidence  of  requirements  database 


Reference  Material: 

Other  Considerations:  Requirements  management  systems  include  tools  such  as  CORE,  DOORS, 
TeamCenter,  etc. 


RG2  Requirements  are  refined,  elaborated,  and  allocated  to  support  design 
or  service(s) 

RG2P1  Establish  and  maintain  design  mission  reference  profiles  that  define 
the  product  characteristics  required  in  engineering  terms  and  document  the 
interaction  of  the  product  with  the  environment,  other  systems  and  operational 
users 


Description:  The  conceptual  set  of  mission  profiles  must  be  refined  into  a  comprehensive  set  of  design 
mission  reference  profiles  that  define  the  product  performance  boundaries,  intended  usage  modes  and 
environments  in  engineering  terms.  Key  characteristics  should  be  traceable  to  specific  mission  scenarios 
with  derived  measures  of  performance.  The  hierarchy  of  technical  performance  parameters  should  be 
extended  to  support  detailed  trade  studies.  Ensure  user  awareness  of  evolving  product  characteristics  and 
changing  mission  scenarios. 

Typical  Work  Products: 


□  Technical  performance  measurements  (TPMs) 

□  Use  cases/mission  profiles 


Reference  Material: 

Other  Considerations:  Consider  anti-tamper  requirements. 


RG2P2  Allocate  the  requirements  to  each  product-component 


Description:  Allocate  requirements,  in  correlation  with  the  top  level  architecture,  to  product  segments  or 
components.  Iteratively  decompose  the  high  level  requirement  into  lower  level  requirements.  Define  key 
interfaces  and  critical  information  exchange  requirements  in  accordance  with  applicable  standards,  and 
where  not  directed,  in  accordance  with  accepted  standards  to  the  extent  practical.  Identify  safety  critical 
functions  and  ensure  affected  hardware  and  software  components  are  treated  as  safety  critical. 

Typical  Work  Products: 


□  Specification/Drawing  tree 
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□  Software  tree 

□  ICWG  charter 

□  DoDAF  views  of  functional  architecture 

□  Functional  Flow  Block  Diagrams 

□  Time  Line  Analysis 

□  Requirements  Allocation  Sheet 

□  ICDs  (Interface  Control  Documents) 

□  ISP  (Information  Support  Plan) 

□  lERs  (Information  Exchange  Requirements) 

Reference  Material:  AFI  63-124 

Other  Considerations:  Key  tools  in  a  functional  analysis  and  allocation  are  Functional  Flow  Block 
Diagrams,  Time  Line  Analysis,  and  the  Requirements  Allocation  Sheet. 


RG3  Iteratively  analyze  and  validate  operational  and  derived  requirements 
throughout  the  product  life  cycle 

RG3P1  Analyze  requirements  to  ensure  that  they  are  necessary  and  sufficient 


Description:  Concisely  and  completely  describe  the  needed  characteristics.  Trace  each  requirement  from 
a  specific  usage  scenario.  Ensure  that  the  technical  requirements  accurately  reflect  the  stakeholder’s 
stated  needs.  Account  for  derived  requirements  that  increase  work  scope  (for  example  in  software)  as  the 
development  proceeds  and  the  design  understanding  evolves.  Establish  and  maintain  a  rationale  for  each 
requirement  to  improve  the  understanding  and  clarity  of  the  need.  Establish  verification  criteria  and 
methods  for  each  requirement.  Evaluate  incremental  development  and/or  potential  requirements  evolution 
over  time.  Compare  Technology  Readiness  Assessments  to  user  requirements. 

Typical  Work  Products: 


□  Technology  Readiness  Assessment  (TRA)  documents 

□  Verification  matrix 

□  System  Requirements  Review  (SRR)  documents 


Reference  Material: 

Other  Considerations:  If  incremental  development  is  used,  control  increment  content  by  allocating  all 
requirements  up-front  to  specific  increments. 


RG3P2  Analyze  requirements  to  balance  stakeholder  needs  and  constraints 

Description:  Establish  and  maintain  the  balance  of  stakeholder  needs  and  project  constraints,  such  as 
performance,  usability,  schedule,  manufacturing,  operational  and  supportability  considerations,  and  life 
cycle  cost.  Perform  trade  studies  using  formal  methods  for  decision  analysis  and  analysis  of  alternatives 
Document  and  obtain  stakeholder  coordination  on  key  decisions  affecting  requirements. 
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Typical  Work  Products: 

□  T rade  studies  with  AoAs 

Reference  Material: 

Other  Considerations:  CAIV  studies  could  be  accomplished.  Consider  Technology  Readiness.  Consider 
an  iterative  process  of  requirements  refinement  involving  the  suppliers  and  the  end  user  as  the  design 
matures.  Ensure  program  baselines  are  consistent  with  estimates  for  the  design  effort  (hardware  and 
software),  cost,  and  schedule,  and  the  disciplined  application  of  effective  systems  and  software  engineering 
processes. 

RG3P3  Validate  requirements  to  ensure  the  evolving  product  will  perform  as 
intended  in  the  operational  environment 


Description:  Requirements  validation  assures  the  technical  requirements  are  the  right  requirements.  The 
goal  is  to  ensure  that  the  product  will  be  operationally  safe,  suitable  and  effective  for  its  intended  use  in  its 
intended  environment.  Requirements  reflect  immediate  operational  needs  and  the  ability  to  support  likely 
future  needs.  Use  M&S  to  the  maximum  extent  feasible.  Ensure  end  user  understanding  and  commitment 
to  the  evolving  requirements. 

Typical  Work  Products: 


□  Traceability  matrix 

□  Successful  SRR  (signed  action  items  &  minutes) 

□  M&S  results 


Reference  Material:  AFI  63-124 

Other  Considerations:  Consider  the  use  of  physical  and  computer  based  simulations  accredited  for  the 
application  by  recognized  authority. 

RG4  Requirements  are  managed  and  controlled,  and  inconsistencies  with 
technical  plans  and  work  products  are  identified 

RG4P1  Use  a  disciplined  process  for  accepting,  vetting,  approving,  and 
providing  requirements  and  changes  to  the  suppliers 


Description:  Requirements  changes  from  unauthorized  sources  that  are  outside  the  flow  of  the  acquirer’s 
established  baseline  management  process  are  not  allowed.  Assess  each  change  to  a  controlled 
requirement  for  impact  to  the  project’s  performance,  cost,  and  schedule  baselines  and  to  project  risk. 
Change  existing  cost,  schedule,  and  performance  baselines  to  accommodate  the  requirements  change. 
Minimize  requirements  creep.  Vet  new  requirements  through  a  standardized  process.  Develop  verification 
methods  as  requirements  are  accepted  into  the  baseline.  Maintain  bi-directional  traceability  between 
system  and  allocated  requirements. 

Typical  Work  Products: 


□  CM  plan 

□  CCB  process  01 

□  Requirements  management  plan 
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□  Requirements  allocation  sheets 

□  Performance  Plan 
Reference  Material:  AFI  63-124 

Other  Considerations:  Performance-based  contracts  should  specify  procedures  or  remedies  for 
reductions  in  price  (or  fee)  when  services  are  not  performed  or  do  not  meet  contract  requirements.  Include 
software  engineering  representation  on  the  requirements  review  board  if  applicable. 

RG4P2  Establish  and  maintain  commitment  to  the  requirements 

Description:  Establish  and  maintain  agreements  with  key  stakeholders  to  manage  expectations  as  the 
product  characteristics  evolve.  Obtain  requirements  commitment  from  the  stakeholders  by  having 
coordinated  and  approved  documents  that  define  requirements  at  multiple  levels. 

Typical  Work  Products: 

□  EMAs  (Expectation  Management  Agreements) 

□  MOAs  and  MOUs 

□  Initial  Capabilities  Document  (ICD) 

□  Product  support  management  plan  (?) 

□  Performance  specifications 

□  Component  specification 
Reference  Material: 

Other  Considerations:  Stakeholders  include  other  government  agencies  such  as  DCMA. 


RG4P3  Establish  and  maintain  bidirectional  traceability  between  requirements 
and  work  products 


Description:  Bidirectional  traceability  ensures  that  all  higher  level  requirements  (and  verification  criteria) 
are  accounted  for  by  the  totality  of  the  lower  level  requirements.  It  also  ensures  that  lower  level 
requirements  are  tied  to  a  parent  requirement  to  prevent  orphan  requirements  at  the  lower  levels. 
Bidirectional  traceability  also  supports  requirements  change  impact  analysis  when  either  high  or  lower  level 
requirements  or  verification  criteria/methods  require  change. 

Typical  Work  Products: 


□  Requirements  traceability  matrix 

□  Requirements  tracking  system 

□  Requirements  correlation  matrix/table 


Reference  Material: 
Other  Considerations: 
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RG4P4  Identify  and  resolve  inconsistencies  between  requirements,  project 
plans,  and  work  products 

Description:  Make  all  technical  plans,  designs,  and  specifications  consistent  with  and  traceable  to 
documented  requirements. 

Typical  Work  Products: 

□  Gap  analysis 

□  Corrective  action  list 

Reference  Material: 

Other  Considerations: 


8.7  Risk  Management  (RM) 

The  purpose  of  Risk  Management  is  to  identify  potential  problems  before  they  occur,  so 
that  risk-handling  activities  may  be  planned  and  invoked  as  needed  across  the  life  of  the 
product  or  project  to  handle  adverse  impacts  on  achieving  objectives. 

Risk  identification  and  estimation  of  probability  of  occurrence  and  impact,  particularly  for 
those  risks  involved  in  meeting  performance  requirements,  schedules,  and  cost  targets, 
largely  determine  the  acquisition  strategy.  The  acquirer  has  a  dual  role:  first,  in  assessing 
and  managing  technical  risks  for  the  duration  of  the  project,  and  second,  in  assessing 
and  managing  technical  risks  associated  with  the  performance  of  the  supplier.  As  the 
acquisition  progresses  to  the  selection  of  a  supplier,  the  risk  specific  to  the  supplier’s 
technical  and  management  approach  then  becomes  important  to  the  success  of  the 
acquisition. 

The  Risk  Management  process  area  can  best  be  described  by  the  following  goals  and 
practices. 

Risk  Management  (RM) 

RMG1  Prepare  for  Risk  Management 

RMG1P1  Determine  risk  sources  and  categories 

Description:  Establish  categories  of  risks  and  risk  sources  for  the  project  initially  and  refine  the  risk 
structure  over  time  (e.g.,  schedule,  cost,  supplier  execution,  technology  readiness,  manufacturing 
readiness,  product  safety,  and  issues  outside  control  of  team),  using  Integrated  Product  Teams.  Quantify 
the  risk  probability  and  consequence  in  terms  of  cost  and  schedule. 

Typical  Work  Products: 


□  Risk  matrix 

□  Risk  management  plan 


Reference  Material:  DoD  Risk  Management  Guide,  AFI  90-901 
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Other  Considerations:  Consider  using  Acquisition  Center  of  Excellence  Risk  Management  Workshops 
when  needed.  For  manufacturing  risks  consider  the  capability  of  planned  production  processes  to  meet 
anticipated  design  tolerances.  Include  the  supplier’s  capacity  and  capabilities  in  the  analysis. 


RMG1P2  Define  the  parameters  used  to  analyze  and  categorize  risks,  and  the 
parameters  used  to  control  the  risk  management  effort 


Description:  Record  the  parameters  used  to  analyze  and  categorize  risks  so  that  they  are  available 
throughout  the  project  period  for  reference  when  circumstances  change  over  time. 

Typical  Work  Products: 


□  Risk  Management  Plan 


Reference  Material: 

Other  Considerations:  Recording  the  parameters  used  to  analyze  and  categorize  risks  allows  the  risks 
to  be  easily  re-categorized  and  analyzed  relative  to  the  original  information  when  changes  occur.  Well 
defined  parameters  provide  common  and  consistent  criteria  for  analyzing,  prioritizing  and  mitigating  the 
risks  to  be  managed. 


RMG1P3  Establish  and  maintain  the  strategy  and  plans  to  be  used  for  risk 
management 


Description:  Identify  risks,  roles,  responsibilities,  and  relationships  of  stakeholders  in  the  risk  management 
process,  schedule  for  addressing  risks,  and  the  plan  to  identify,  assess,  track,  and  handle  (control/mitigate, 
avoid,  assume  or  transfer)  risks.  Develop  a  Risk  Management  Plan  (RMP)  (may  be  developed  jointly  by  the 
government,  contractor,  and  suppliers).  Update  and  use  the  RMP  throughout  the  system’s  life  cycle. 

Typical  Work  Products: 


□  Risk  Management  Plan 

□  Systems  Engineering  Plan 

Reference  Material:  Defense  Acquisition  University  Risk  Management  Guide  for  DOD  Acquisition,  AFMC 
Risk  Pamphlet,  DoD  Risk  Management  Guide,  MIL-STD-882,  AFI  91-202,  DoDI  5000.02,  Directive  Type 
Memorandum  (DTM-08-048)  SCRM  to  Improve  the  Integrity  of  Components  Used  in  DoD 
Systems 

Other  Considerations:  Consider  performing  periodic  integrated  risk  assessments.  Safety  related  risks  are 
managed  in  accordance  with  AFI  91-202.  Include  in  the  overall  risk  strategy  the  Supply  Chain 
Risk  Management  (SCRM)  approach.  SCRM  includes  both  trusted  key  components  and 
software  or  firmware  that  is  free  of  malicious  code.  Define  a  process  and  criteria  for  escalating 
risks  to  the  next  higher  management  level.  Establish  a  threshold  for  risk  acceptance. 

RMG2  Identify  and  analyze  risks  to  determine  their  relative  importance 

RMG2P1  Identify  and  document  the  technical  risks 


Description:  Document  program,  technology,  and  manufacturing  risks  in  a  concise  statement  that  includes 
the  sources,  context,  conditions  and  consequences  of  risk  occurrence.  The  identification  of  issues, 
hazards,  threats,  and  vulnerabilities  that  could  negatively  affect  work  efforts  or  plans  is  the  basis  for  sound 
and  successful  risk  management.  Establish  and  maintain  a  failure  mode  and  effects  analysis  to 
characterize  how  the  product  or  manufacturing  processes  can  fail. 
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Typical  Work  Products: 


□  Risk  matrix 

□  FMECA 

□  Hazard  Analysis 


Reference  Material:  MIL-STD-1629A 

Other  Considerations:  Risks  must  be  identified  and  described  before  they  can  be  analyzed  and 
managed  properly.  Consider  supportability  risks  in  hardware,  software  and  firmware.  Pay  particular 
attention  to  external  factors  which  could  influence  a  product  acquisition  and  sustainment  over  which  the 
team  may  have  little  control. 


RMG2P2  Evaluate  and  categorize  each  identified  risk  using  the  defined  risk 
categories  and  parameters,  and  determine  its  relative  priority 

Description:  Assign  relative  importance  to  each  identified  risk,  and  use  it  to  determine  when  appropriate 
management  attention  is  required.  Aggregate  risks  based  on  their  interrelationships,  and  develop  options 
for  all  risks  within  the  category.  Establish  and  maintain  a  criticality  analysis  to  chart  the  probability  of  failure 
modes  against  the  severity  of  their  consequences. 

Typical  Work  Products: 


□  Risk  matrix 

□  FMECA 

□  Hazard  Analysis 


Reference  Material:  MIL-STD-1629A 

Other  Considerations:  When  an  aggregate  risk  is  formed  by  a  roll  up  of  lower  level  risks,  care  must  be 
taken  to  ensure  that  important  lower  level  risks  are  not  ignored.  The  acquirer  should  assess  project  risks 
both  independently  and  in  conjunction  with  the  supplier’s  risk  management  procedures. 


RMG3  Perform  risk  handling  to  manage  adverse  impacts  on  the  project 

RMG3P1  Establish  and  maintain  plans  for  mitigating  each  of  the  important  risks 
to  the  project 


Description:  Develop  an  approach  for  handling  each  risk  using  the  appropriate  method  and/or  technology 
insertion.  Risk  handling  methods  include:  avoidance;  assumption;  transfer;  and  control/mitigation. 

Evaluate  risk  handling  plans  for  impact  on  all  aspects  of  the  project  and  ensure  inclusion  of  risk  mitigation 
plans  in  the  Integrated  Master  Schedule.  Document  the  plans,  and  coordinate  with  all  stakeholders.  Identify 
and  assess  alternative  technologies  for  substitution  in  place  of  unacceptable  higher  risk  technologies. 

Typical  Work  Products: 


□  Risk  transfer,  assumption,  avoidance,  or  mitigation  plan 

□  Systems  Engineering  Plan 

□  Integrated  Master  Schedule 
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Reference  Material: 

Other  Considerations:  After  identifying  potential  technology  risks  a  project  might  develop  an  incremental 
approach  for  technology  insertion  later  in  the  life  cycle  while  using  alternative  lower  risk  technologies  for  the 
initial  product  units. 


RMG3P2  Monitor  and  assess  risk  handling  activities 


Description:  Continually  monitor  risks  and  risk  handling,  and,  when  warranted,  adjust  risk  handling  or 
mitigation  plans.  Establish  a  method  to  periodically  review  each  risk  and  provide  current  status  to 
stakeholders.  Various  risk  handling  paths  may  affect  the  project  in  ways  difficult  to  foresee.  Execute  risk 
handling,  including  mitigation  actions  promptly. 

Typical  Work  Products: 


□  Risk  transfer,  assumption,  avoidance,  or  mitigation  plan  results 


Reference  Material: 
Other  Considerations: 


8.8  Transition,  Fielding,  &  Sustainment  (TFS) 

The  purpose  of  the  Transition,  Fielding,  &  Sustainment  process  is  to  prepare  for  and 
execute  the  support,  maintenance,  repair,  and  disposal  of  a  product  while  ensuring  it  is 
safe,  suitable,  and  effective  while  it  is  fielded  and  operated. 

Sustainment  is  the  planning,  programming,  and  executing  of  a  support  strategy.  It 
includes  specific  activities  in  all  phases  of  a  product  lifecycle  from  product  concept 
formulation  to  demilitarization  and  disposal. 

The  Transition,  Fielding,  &  Sustainment  process  area  can  best  be  described  by  the 
following  goals  and  practices. 

TFSG1  Prepare  to  support  operations,  maintenance,  repair,  and  disposal  of 
the  product. 

TFSG1P1  Establish  and  maintain  plans  for  logistics  support  of  the  product. 


Description:  Establish  and  maintain  initial  and  life-cycle  resources  requirements  for  performing  operations 
and  support.  Establish  and  maintain  mechanisms  for  ensuring  the  involvement  of  support  organizations 
throughout  product  acquisition.  Identify  the  policies  which  must  be  implemented  to  manage  the  failure 
modes  which  could  cause  the  functional  failure  of  the  product  in  operations  (e.g.,  reliability  centered 
maintenance). 

Typical  Work  Products: 


□  Integrated  Logistics  Support  Plan 

□  Life  Cycle  Management  Plan  (LCMP) 

□  Item  Unique  Identification  (IUID)  Plan 
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□  Depot  Maintenance  Activation  Plan  (DMAP)  or  Source  of  Repair  Assignment  Process  (SORAP) 

□  Training  plans 

□  Packaging,  Handling,  Storage  and  Transportation  (PHS&T)  documentation  (see 
LHA  reference) 


Reference  Material:  AFI  63-101  ,  AFI  63-1201,  DoDI  5000.02,  AFI  63-1101,  AFI  63-107,  Logistics  Health 
Assessment  (LHA)  Process  Guide 

Other  Considerations:  Plans  should  consider  maintenance,  manpower  &  personnel,  supply  support, 
support  equipment,  technical  data,  training  &  training  support,  computer  resource  support,  facilities, 
packaging,  handling,  storage  and  transportation.  Identify  data  rights  and  integrate  with  support  strategy 
options  (see  LHA  reference). 


TFSG1P2  Establish  and  maintain  the  strategy  and  plan(s)  for  transitioning  acquired 
products  into  operational  use  and  support. 


Description:  Address  objectives  and  scope  of  the  transition,  identification  and  responsibilities  of  the 
support  organizations,  resource  requirements,  criteria  for  transition  from  the  development  to  the 
sustainment  life  cycle  phase  and  related  changes  in  authority  and  responsibility,  definition  of  transition 
activities,  schedule,  installations  of  products  to  be  delivered,  warranty  and  data  rights  provisions 

Typical  Work  Products: 


□  Transition  support  plan 

□  Depot  Source  of  Repair  (DSOR)  package  (Source  of  Repair  Assignment,  Depot  Maintenance  Inter¬ 
service,  Strategic  Source  of  Repair) 

□  Life  Cycle  Management  Plan 


Reference  Material:  AFI  63-101,  Acquisition  Sustainment  Tool  Kit 

Other  Considerations: 


TFSG1 P3  Establish  and  maintain  plan(s)  for  the  disposal  of  the  product. 


Description:  Document  disposal  methodology  of  any  existing  products  to  be  replaced  as  well  as  eventual 
disposal  of  the  new  product. 

Typical  Work  Products: 


□  Disposal  plan 


Reference  Material: 

Other  Considerations:  Consider  SMR  (Source,  Maintenance,  Repair)  codes  and  DRMO  (Defense 
Revitalization  &  Marketing  Office). 


TFSG2  Ensure  the  resources,  capacity  and  capability  to  support  the  operations, 
maintenance,  repair,  and  disposal  of  the  product  are  ready  prior  to  need. 
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TFSG2P1  Establish  and  maintain  budgets  for  sustainment  activities. 


Description:  Ensure  that  sustainment  organizations  have  established  funding  (e.g.  amounts  and  sources) 
that  complement  ongoing  acquisition  funding  and  are  consistent  with  suppliers’  warranties,  product 
deliveries  and  plans  for  supporting  the  product. 

Typical  Work  Products: 


□  POM  input 


Reference  Material:  AFI  21-101,  AFI  38-201 
Other  Considerations: 


TFSG2P2  Establish  and  maintain  processes  and  procedures  for  repair,  overhaul, 
or  modification. 


Description:  Ensure  product  technical  documentation  for  repair,  overhaul,  or  modification  and 
associated  training  for  support  personnel  is  complete  and  verified. 

Typical  Work  Products: 

□  Technical  orders/manuals  (see  LHA  reference) 

□  Work  process  orders 

□  Drawings 

□  Procedures/process  flow  charts 

□  Work  Control  Documents  (WCD) 


Reference  Material:  AFPD  23-1,  T.O.  00-5-1,  AFI  21-303,  AFI  21-101,  Logistics  Health  Assessment 
(LHA)  Process  Guide 

Other  Considerations:  This  documentation  complements  the  technical  data  developed  during  design 
and  manufacturing.  Address  qualification  and  approval  of  special  processes  up  front  and  early.  All 
process  documentation  affecting  product  quality  should  be  placed  under  configuration  control. 


TFSG2P3  Ensure  readiness  for  fielding  and  transition  to  operations  and  support. 


Description:  Identify  and  monitor  contract  requirements  for  support  products  and  services.  Conduct 
Provisioning  Conferences  and  perform  associated  planning  to  support  sustainment.  Identify  and  monitor 
requirements  for  initial  spares,  support  equipment,  facilities  and  facility  modifications.  Establish  and 
maintain  criteria  and  agreements  for  transition  to  operations  and  support.  Ensure  qualified  suppliers 
continue  to  exist  for  parts  and  materials. 

Typical  Work  Products: 


□  Contract 

□  Spares  list 
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□  Sustainment  planning 

□  DMSMS  plan 

□  Packaging  Handling,  Storage,  &  Transportation  (PHS&T)  documentation  (PHS&T  available  in 
support  of  provisioning  plan) 


Reference  Material:  AFI  21-101,  SD22 

Other  Considerations:  Stakeholders  should  participate  in  Provisioning  Conferences  and  associated 
planning  to  ensure  appropriate  sustainment  support  will  be  available  and  timely. 


TFSG2P4  Ensure  product  support  is  maintained  during  transition. 


Description:  Activate  the  product  in  its  intended  operational  situation  and  monitor  product  operations  to 
ensure  stakeholder  satisfaction.  Analyze  the  results  of  all  transition  activities  and  identify  appropriate 
action. 

Typical  Work  Products: 


□  Fielded  performance  data 

□  PHS&T  documentation  (see  LHA  reference) 


Reference  Material:  Logistics  Health  Assessment  (LHA)  Process  Guide 

Other  Considerations:  Fielded  system  performance  data  can  be  obtained  from  maintenance  systems 
such  as  REMIS,  CAMS,  DRILS  etc.  and  maintenance  reports,  accident  investigation  reports,  and  supplier 
databases.  Execute  Packaging,  Handling,  Storage,  and  Transportation  (PHS&T)  if  appropriate. 


TFSG2P5  Establish  and  maintain  the  required  facilities,  manpower,  tooling  and 
test  equipment  for  repair,  overhaul,  or  modification 


Description:  Define  requirements  for  support  equipment,  facilities  and  facility  modifications.  Identify 
required  manpower  (quantity  and  type)  and  personnel  qualifications  (education,  training,  skills  and 
experience). 

Typical  Work  Products: 


□  Facilities  plan 

□  Staffing  plan 

□  Test  equipment/tooling  list 

□  Training  material 

□  Life  Cycle  Sustainment  Plan  (LCSP) 


Reference  Material:  AFI  63-101,  AFI  21-101,  AFI  36-2232 

Other  Considerations:  Infrastructure  considerations  include  buildings,  workspace,  associated  utilities, 
tooling,  capital  equipment,  and  supporting  services  such  as  transport,  communication  or  special  handling. 


TFSG3  Repair,  overhaul,  or  modify  the  product 
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TFSG3P1  Repair,  overhaul  or  modify  the  product  in  accordance  with  established 
procedures  and  processes. 


Description:  Ensure  that  repair,  overhaul  or  modification  operations  are  carried  out  in  accordance  with 
approved  technical  data.  Establish  monitoring,  measurement,  analysis,  and  improvement  processes  to 
ensure  conformity  of  the  product,  with  feedback  mechanisms  for  continued  quality. 

Typical  Work  Products: 


□  List  of  tools,  special  inspection  equipment,  test  equipment  etc.  and  instructions  associated  with 
their  use 

□  Critical  process  procedures 

□  Test  plans/procedures 

□  Drawings 

□  Parts  Lists 

□  Process  documents 

□  Inspection  documents 


Reference  Material:  AFI  63-501,  AFI  63-1101,  T.O.  00-5-1 

Other  Considerations:  The  use  of  statistical  process  control  is  appropriate  for  high-volume  processing. 
Incorporate  Reliability  Centered  Maintenance  (RCM)  concepts  to  determine  level  and  type  of  maintenance 
needed.  Establish  a  preventative  maintenance  program  based  on  the  principles  of  RCM  which  is  the 
optimum  mix  of  reactive,  proactive,  time  or  interval-based,  and  condition  based  maintenance  practices  to 
maximize  system  reliability  while  minimizing  life-cycle  costs.  In  implementing  a  particular  type  of 
maintenance  practice,  consider  the  probability  and  consequence  of  failure,  available  historical  data,  risk 
tolerance  and  resource  availability. 


TFSG3P2  Establish  and  maintain  inventory  and  supplier  management/control  to 
execute  the  repair,  overhaul  or  modification 


Description:  Develop  procedures  and  guidelines  for  evaluating  and  selecting  qualified  manufacturers. 
Establish  feedback  mechanisms  to  ensure  continued  quality,  including  review  of  product  documentation, 
periodic  inspections  and  audits  at  supplier’s  premises,  and  measurement  systems.  Establish  agreements 
with  the  supplier  for  notification  and  if  necessary,  acquirer  approval  of  nonconforming  product/material  and 
right  of  access  by  acquirer  to  all  applicable  data  and  records  related  to  purchased  products/materials. 
Identify  and  negotiate  repair  and  overhaul  rates. 

Typical  Work  Products: 


□  Qualified  supplier  list 

□  Certificate  of  conformity 

□  Supplier  rating 

□  Contract 
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Reference  Material:  Federal  Acquisition  Regulation  (FAR),  AFI  63-501 

Other  Considerations: 


TFSG4  Maintain  Operational  Safety,  Suitability,  and  Effectiveness  (OSS&E) 
TFSG4P1  Establish  and  maintain  OSS&E  baseline(s). 


Description:  Establish,  maintain  and  document  OSS&E  baseline(s)  in  an  OSS&E  Baseline  Document 
(OBD)  during  operational  use  with  stakeholder  coordination.  Develop  mechanisms  to  ensure  conformity  to 
the  OBD  (i.e.,  health  monitoring  and  testing  of  fielded  products  carried  out  with  results  documented  and 
compared  to  the  appropriate  baseline  measures).  Take  action  to  analyze  and  correct  deficiencies,  If 
results  do  not  compare  favorably.  Maintain  all  required  certifications.  Document  procedures  for 
coordination  of  OSS&E  assurance  among  related  products. 

Typical  Work  Products: 


□  Surveillance  test  plan 

□  Weapon  Systems  Evaluation  Program 

□  OBD 


Reference  Material:  AFI  63-1201,  AFI  63-101,  TO  00-35D-54  -  Joint  Deficiency  Reporting  System 
(JDRS) 

Other  Considerations: 


TFSG4P2  Identify  and  monitor  safety  critical  items 


Description:  Establish  and  maintain  a  disciplined  process  for  handling  of  safety  critical  items,  including 
identification  of  established  technical  authorities,  quality  assurance,  and  qualified  manufacturers. 

Typical  Work  Products: 


□  Safety  Critical  Items  and/or  Critical  Safety  Items  list 
Reference  Material:  AFMAN  23-110,  Aviation  Critical  Safety  Item  Management  Handbook,  AFI  63- 
1201,  MIL-STD-882  (DoD  Standard  Practice  for  System  Safety),  AFI  20-106  (Management  of  Aviation 
Critical  Safety  Items) 

Other  Considerations: 

TFSG4P3  Identify  and  mitigate  hazards 


Description:  Establish  and  maintain  a  hazards  tracking  database.  Ensure  high/serious  hazards  are 
mitigated  or  accepted  by  appropriate  authority.  Establish  and  maintain  a  process  for  handling  accidents 
involving  the  product. 

Typical  Work  Products: 


□  Hazards  list 

□  Mishap 
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□  Hazard  Analysis 

Reference  Material:  AFMCPAM  91-104,  AFI  63-1201 

Other  Considerations: 

TFSG4P4  Identify  and  monitor  operations  and  maintenance  data 

Description:  Establish  mechanisms  for  collecting,  reviewing  and  acting  upon  data  related  to  product 
operations  and  support  (e.g.  defects,  flight  hours,  anomalies).  Monitor  reliability,  maintainability, 
supportability  data,  as  well  as  that  for  the  other  Integrated  Logistics  Support  (ILS)  areas. 

Typical  Work  Products: 

□  Databases 

Reference  Material:  AFI  63-501,  AFI  21-101,  TO  00-35D-54  -  Joint  Deficiency  Reporting  System  (JDRS) 

Other  Considerations:  Maintain  awareness  of  system  performance  against  stakeholder  collaborated 
availability  requirements.  One  approach  is  to  consider  usage  of  Core  Automated  Maintenance  System 
(CAMS). 

TFSG4P5  Execute  (as  required)  the  plan  for  decommissioning  and  disposal  of 
the  product 

Description:  Identify  and  monitor  requirements  for  system  disposal  or  migration  to  long  term  storage. 

Typical  Work  Products: 

□  Disposal  plan 

Reference  Material:  AFI  16-402,  AFI  63-101 

Other  Considerations:  This  only  applies  to  programs  that  have  actually  disposed  products,  components, 
etc. 


8.9  Technical  Management  &  Control  (TMC) 

The  purpose  of  Technical  Management  and  Control  (TMC)  is  to  provide  an 
understanding  of  the  project’s  technical  progress  so  that  appropriate  corrective  actions 
can  be  taken  when  the  project’s  performance  deviates  significantly  from  the  plan.  TMC 
provides  for  establishing  and  managing  the  project  and  the  involvement  of  relevant 
stakeholders  following  the  principles  of  Integrated  Product  and  Process  Development 
(IPPD).  It  also  allows  for  developing  and  sustaining  the  infrastructure  and  measurement 
capability  to  support  management  information  needs. 

A  project’s  documented  plan  is  the  basis  for  monitoring  activities,  communicating  status, 
and  taking  corrective  action.  Progress  is  primarily  determined  by  comparing  actual  work 
product  and  task  attributes,  effort,  cost,  and  schedule  to  the  plan  at  prescribed  milestones 
or  control  levels  in  the  project  schedule  or  WBS.  Appropriate  visibility  of  progress  enables 
timely  corrective  action  to  be  taken  when  performance  deviates  significantly  from  the 


Version  2-21  September  2010 


61 


Version  2-21  September  2010 


plan.  A  deviation  is  significant  if,  when  left  unresolved,  it  precludes  the  project  from 
meeting  its  objectives. 

The  term  project  plan  is  used  throughout  these  practices  to  refer  to  the  overall  plan  for 
controlling  the  project. 

Monitoring  and  control  functions  are  established  early  in  the  project  as  the  project’s 
planning  is  performed  and  the  acquisition  strategy  is  defined.  As  the  acquisition  of 
technology  solutions  unfolds,  monitoring  and  control  activities  are  essential  to  ensure  that 
appropriate  resources  are  being  applied  and  that  acquirer  activities  are  progressing 
according  to  plan. 

When  actual  status  deviates  significantly  from  expected  values,  corrective  actions  are 
taken,  as  appropriate.  These  actions  may  require  replanning,  which  may  include  revising 
the  original  plan,  establishing  new  agreements,  or  including  additional  mitigation  activities 
in  the  current  plan. 

If  corrective  action  is  required  to  resolve  variances  from  project  plans,  these  actions 
should  be  defined  and  tracked  to  closure. 

The  Technical  Management  &  Control  process  area  can  best  be  described  by  the 
following  goals  and  practices. 

TMCG1  Prepare  for  Integrated  Management 


TMCG1P1  Establish  and  maintain  the  project  environment. 


Description:  Define  the  project’s  processes  to  form  an  integrated  approach  for  managing  the  product 
throughout  its  lifecycle.  Implement  technical  standard  work  and  process  guidelines.  Establish  a  work 
environment  which  facilitates  work  by  co-located  or  distributed  teams. 

Typical  Work  Products: 


□  Standard  Operating  Procedures  (Ols,  guides,  etc.) 


Reference  Material:  DoDI  5000.02 

Other  Considerations:  Consider  common  collaborative  tools  for  communication  and  development  (e.g. 
DOORS,  Net  Meeting,  MS  Project,  Live  Link,  integrated  development  environment  (IDE),  Communities  of 
Practice  (CoP)  etc.) 


TMCG1P2  Establish  and  maintain  supplier  agreements. 


Description:  Evaluate  and  recommend  optimal  acquisition  type  (e.g.,  COTS,  Formal  Agreement,  In-House 
Vendor,  etc)  for  each  product  or  product  component  to  be  acquired.  Establish  technical  requirements  and 
provide  support  to  contracting  agency  to  select  suppliers  and  establish  supplier  agreements  (e.g.,  SOW, 
Contract,  MOA). 
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Typical  Work  Products: 


□  Acquisition  strategy 

□  Instructions  to  offerors 

□  Technical  evaluation  areas,  factors  and  criteria 

□  Supplier  agreements  or  subcontracts 

□  Statement  of  Objectives  (SOO) 

□  Program  Statement  of  Work 

□  List  of  compliance  standards  and  specifications 

□  Contract  Data  Requirements  List  (CDRL) 

□  Request  for  Proposal  Package 


Reference  Material: 

DoD  Guide  for  Contracting  for  SE,  Defense  Acquisition  Guide 

Other  Considerations:  Ensure  supplier  agreements  provide  the  project  team  insight  into  key 
supplier/subcontractor  selection  and  their  processes/activities.  Include  suppliers  (below  prime  contractor 
level)  in  the  IPPD  construct.  Any  supplier  processes  that  are  critical  to  the  success  of  the  project  should  be 
identified  in  the  supplier  agreement  and  monitored  during  execution.  In  cases  of  low  risk  it  is  possible  that 
no  processes  are  formally  monitored.  For  software-intensive  systems,  identify  software  as  a  separate  sub¬ 
factor  for  supplier  evaluation.  Ensure  that  the  supplier  commits  to  their  Software  Development  Plan  (SDP) 
as  the  controlling  plan  for  software  processes.  Ensure  that  planned  software  development  effort  and 
schedule  is  consistent  with  the  overall  program  budget  and  development  schedule. 


TMCG1P3  Establish  and  maintain  integrated  product  teams  (IPT). 

Description:  Establish  a  shared  vision  for  the  project  consistent  with  the  mission,  goals,  expectations,  and 
constraints  with  stakeholder  participation.  Establish  and  maintain  integrated  teams  based  on  a  product 
oriented  hierarchy.  Allocate  requirements,  responsibilities,  tasks  and  interfaces  to  teams  in  the  integrated 
team  structure.  Document  the  team  membership,  its  leaders,  resources  and  other  constraints. 

Typical  Work  Products: 


□  IPT  Charter 

□  IPT  minutes 


Reference  Material:  DoD  IPPD  document 

Other  Considerations:  Considerations  should  include  stakeholder  turnover  and  periodic  team  health 
checks  to  verify  if  team  members  are  fulfilling  their  established  and  agreed-upon  roles  and  responsibilities. 


TMCG1P4  Establish  and  maintain  measurement  approach 


Description:  Establish  measurement  objectives  and  measurement  criteria  based  on  projects  risks  and 
issues.  Specify  how  measurement  data  will  be  obtained,  stored,  updated  and  presented.  Specify 
measures  and  analysis  procedures  to  address  information  needs.  Determine  the  frequency  of  data 
measurement  by  consideration  of  project  maturity/complexity  and  stakeholder  needs. 

Typical  Work  Products: 
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□  Measurement  Objectives 

□  TPMs 

□  EVM 

□  Software  measurements 


Reference  Material:  Air  Force  Weapon  Systems  Software  Management  Guidebook 


Other  Considerations:  Appropriate  information  must  be  identified  in  order  to  develop  and  sustain  the 
measurement  capability  that  is  used  to  support  management  information  and  decision-making  needs. 
Attention  to  storage  and  retrieval  procedures  helps  ensure  that  data  are  available  and  accessible  for  future 
use.  Air  Force  core  software  metrics  (size,  effort,  schedule,  quality,  requirements  definition  and  stability, 
staffing,  progress,  and  computer  resources  utilization)  should  be  used  to  track  all  software  to  be  developed, 
integrated,  and  delivered.  Core  metrics  should  be  supplemented  with  additional  software  metrics  as 
appropriate,  including  emphasis  on  complexity  of  the  software  to  be  developed,  staff  capability  and 
productivity  rather  than  simply  quantity,  etc.  Measurement  and  analysis  applies  to  tracking  the  progress  of 
the  acquisition  team  as  well  as  the  progress  of  the  supplier. 


TMCG2  Perform  Integrated  Management. 

TMCG2P1  Monitor  and  control  the  project  in  accordance  with  project 
commitments. 


Description:  Monitor  and  control  performance  of  agreements  on  commitments  addressing  each  critical 
dependency  with  those  responsible  for  providing  or  receiving  the  work  product 

Typical  Work  Products: 

□  Review  report 

□  List  of  critical  dependencies 

□  Status  of  critical  dependencies 

□  MO  As 

□  GFP  list 

□  Interface  list 


Reference  Material: 

Other  Considerations:  Deliverables  from  an  external  project  (e.g.  government  furnished  property) 
necessary  to  comply  with  program  schedule.  Outgoing:  project  deliverables  (e.g.  equipment  or  data)  to 
other  projects  that  depend  upon  those  deliverables  to  meet  their  schedules.  Think:  System  of  Systems  and 
Service  Oriented  Architecture  (SOA)  services. 

TMCG2P2  Monitor  &  control  coordination  and  collaboration. 


Description:  Manage  stakeholder  involvement.  Resolve  coordination  issues.  Ensure  collaboration  among 
interfacing  teams.  Monitor  Stakeholder  involvement. 
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Typical  Work  Products: 

□  MOAs, 

□  Minutes 
Reference  Material: 
Other  Considerations: 


TMCG2P3  Execute  Supplier  Agreements. 


Description:  Monitor  and  accomplish  technical  activities  as  specified  in  supplier  agreements.  Monitor 
selected  supplier  processes.  Evaluate  and  accept  the  supplier  products. 

Typical  Work  Products: 


□  Status  Reports 

□  Design  review  slides  and  minutes 

□  Process  review  and  discrepancy  reports 

□  Receiving  reports 


Reference  Material: 

Other  Considerations:  Perform  technical  activities  with  supplier(s)  as  described  in  the  agreement(s).  This 
typically  includes  periodic  and  event  driven  technical  interchange  meetings,  and  periodic  status  reports. 
Any  processes  that  were  identified  for  monitoring  should  be  analyzed  to  proactively  identify  serious  issues. 
Acceptance  reviews,  tests,  and  configuration  audits  should  be  accomplished  before  accepting  the  product 
as  defined  in  the  supplier  agreement(s).  Assess  how  well  the  supplier  is  adhering  to  their  documented 
development  and  sustainment  processes  and  evaluate  the  effectiveness  of  their  defined  processes.  The 
assessments  may  be  conducted  as  informal  reviews  or  independent  assessments. 


TMCG2P4  Obtain  and  analyze  specified  measurement  data. 


Description:  Collect  and  analyze  measurement  data  as  planned,  conduct  additional  analysis  as 
necessary,  review  results  with  relevant  stakeholders,  and  refine  measurement  criteria  for  future  analysis. 
Monitor  and  analyze  progress  against  technical,  cost,  and  schedule  plans  immediately  after  they  are 
established  and  throughout  the  life  cycle.  Analyze  root  causes  of  variances. 

Typical  Work  Products: 


□  EVMS  reports 

□  Variance  analysis  reports 

□  Software  metrics  reports 

Reference  Material:  Department  of  Defense.  Practical  Software  and  Systems  Measurement  Guidebook. 
www.psmsc.com,  ANSI/EIA-748,  Defense  Acquisition  Guidebook  Chapter  11,  and  the  DoD  Earned  Value 
Management  Implementation  Guide  (Oct  06) 
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Other  Considerations: 

TMCG2P5  Monitor  the  development  and  delivery  of  project  data. 

Description:  Review  deliverable  technical  data  for  completeness,  accuracy,  and  currency  to  ensure 
program  integrity  and  health.  Review  the  process  for  developing  the  project  data. 

Typical  Work  Products: 

□  Records  of  data  management  reviews 

□  Records  of  supplier  data  management 

□  Drawing  release  process 

Reference  Material: 

Other  Considerations: 


TMCG3  Monitor  &  control  technical  progress 

TMCG3P1  Technical  reviews  and  audits  are  conducted  when  all  the  key  entry 
criteria  are  met  and  closed  when  the  exit  criteria  are  met. 

Description:  Plan  and  conduct  technical  reviews,  assessments,  and  audits  in  accordance  with  the 
IMP/IMS  using  the  applicable  standards  and  regulations. 

Typical  Work  Products: 

□  Review  minutes 

□  Action  items 

□  Records 

Reference  Material:  Defense  Acquisition  Guide 

Other  Considerations:  Projects  can  utilize  independent  review  teams  as  warranted. 


TMCG3P2  Assess  results  of  technical  reviews  to  support  key  milestone 
decisions,  higher  level  reporting,  and  project  re-planning  as  required 

Description:  Self-explanatory 

Typical  Work  Products: 

□  pops 

□  SMART 


Version  2-21  September  2010 


66 


Version  2-21  September  2010 


□  DAES 

□  SARS 

□  Milestone  decision  review  minutes 

□  PMR  documentation 

□  Progress  review  minutes 

Reference  Material: 

Other  Considerations:  Results  of  technical  reviews  may  drive  a  change  in  project  direction. 


TMCG3P3  Manage  project  work  products  and  data 

Description:  Manage  project  data,  work  products  and  documents  with  the  appropriate  levels  of  version 
control.  Monitor  the  established  mechanisms  to  ensure  project  data  are  controlled. 

Typical  Work  Products: 

□  SEP  approval  procedures 

□  IDE  version  control  policy 

□  Contract  change  process 

□  Integrated  baseline  review 

□  LCMP  approval  procedures 

Reference  Material: 

Other  Considerations: 

TMCG4  Monitor  and  Control  Corrective  Actions 

TMCG4P1  Collect  and  analyze  the  project  issues  to  determine  and  track 
corrective  actions 

Description:  Collect  and  analyze  project  issues  through  program  management  reviews,  technical  reviews, 
milestone  reviews,  and  analysis  of  program  metrics.  Perform  corrective  actions  as  required  by  modifying 
the  product;  statement  of  work/objectives;  modifying  requirements/specifications;  revising  estimates  and 
plans;  renegotiating  commitments;  adding  resources;  changing  processes;  or  revising  mitigation  actions. 

Typical  Work  Products: 

□  Engineering  change  proposals 

□  Contract  change  proposals 

□  Trade  Studies 
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Reference  Material: 

Other  Considerations:  Corrective  action  should  be  determined  in  sufficient  time  to  properly  resolve  the 
issue. 


TMCG4P2  Establish  and  maintain  a  deficiency  reporting  system 


Description:  A  deficiency  reporting  information  system  is  established  and  maintained  that  includes 
deficiency  identification,  root  cause  analysis,  and  corrective  action  mechanisms.  Establish  and  maintain 
criteria,  processes,  and  procedures  for  determination  and  disposition  of  nonconforming  products  to  prevent 
their  unintended  use  or  delivery. 


Typical  Work  Products: 


□  Deficiency  Reports 

□  Deficiency  reporting  system 

□  Corrective  Action  reports 


Reference  Material:  T.O.  00-35D-54,  AFI  21-104,  AFI  21-118,  AFI  63-501,  AFI  63-1201 

Other  Considerations:  An  automated  defect  reporting/tracking  system  can  enhance  timely  analysis  of 
problem  areas  and  verification  of  effectiveness  of  action  taken  to  correct  problems.  A  corrective  action 
team  should  be  established  to  ensure  adequate  attention  to  the  causes  of  defects. 


TMCG4P3  Manage  corrective  actions  to  closure. 


Description:  Monitor  and  manage  corrective  actions  to  closure,  identifying  and  collecting  lessons  learned 
to  be  used  to  update  processes. 

Typical  Work  Products: 


□ 


Reference  Material: 

Other  Considerations:  Management  to  closure  includes  analyzing  results  of  corrective  actions  to 
determine  their  effectiveness;  determining  and  documenting  appropriate  actions  to  correct  deviations;  and 
documenting  lessons  learned  to  support  future  planning  and  risk  management  activity. 


8.10  Verification  and  Validation  (V) 

The  purpose  of  Verification  is  to  ensure  that  work  products  meet  their  specified 
requirements.  The  purpose  of  Validation  is  to  demonstrate  that  a  product  or  product 
component  fulfills  its  intended  use  when  placed  in  its  intended  environment. 

Verification  and  validation  methods  (e.g.,  peer  reviews,  modeling  and  simulation)  can  be 
applied  to  work  products  (e.g.,  designs  and  prototypes)  as  well  as  to  the  product  and 
product  components.  The  work  products  should  be  selected  based  on  which  are  the  best 
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predictors  of  how  well  the  delivered  end  product  and  product  components  will  satisfy  user 
needs. 

The  acquirer  should  ensure  that  a  proper  verification  environment  exists,  that  it  selects 
work  products  to  evaluate  based  on  documented  criteria,  and  that  the  supplier  uses 
appropriate  methods  to  verify  its  work  products.  In  this  context,  the  test  and  evaluation 
community  is  a  major  stakeholder,  and  should  participate  in  up-front  planning  through 
final-product  acceptance. 

The  supplier  and/or  the  test  community  may  perform  many  of  these  practices,  with  the 
acquirer  facilitating  the  correction  of  deficiencies  or  enhancements  by  the  supplier  or 
follow-on  maintenance  organization.  It  is  important  that  the  acquirer  define  at  the  outset 
the  degree  to  which  verification  and  validation  are  required  both  early  in  the  definition  of 
the  project  and  later  when  the  products  are  received.  Test  and  analysis  techniques 
should  be  implemented  as  early  as  possible  to  identify  deficiencies  that  require  corrective 
action  to  meet  system  requirements. 

Product  verification  activities  are  routinely  conducted  throughout  the  entire  contract 
performance  period,  and  results  are  analyzed  to  determine  acceptability  of  the  products. 
Validation  activities  are  normally  performed  early  and  continuously  throughout  the 
acquisition  life  cycle.  Product  validation  activities  can  be  applied  to  all  aspects  of  the 
product  in  any  of  its  intended  environments,  such  as  operation,  training,  manufacturing, 
maintenance,  and  support  services. 

The  Verification  and  Validation  process  area  can  best  be  described  by  the  following  goals 
and  practices. 

VG1  Prepare  for  verification 

VG1P1  Establish  and  maintain  the  overall  verification  strategy  and  plan, 
including  integrated  testing  approach 


Description:  Address,  in  the  verification  strategy,  the  technical  approach  to  product  verification  and  the 
specific  approaches  that  will  be  used  to  verify  that  work  products  meet  their  specified  requirements. 
Establish  verification  criteria  and  methods  for  each  requirement.  Proactively  seek  opportunities  for 
combined  testing  with  the  operational  test  agencies  and  oversight  offices  and  form  an  Integrated  Test 
Team. 

Typical  Work  Products: 


□  Document  modeling,  simulation,  ground  and  flight  test  activities  planned  for  product  verification  in 
the  Test  and  Evaluation  Master  Plan  (TEMP)  or  similar  document  if  a  TEMP  does  not  exist. 

□  Use  Cases  are  used  to  define  test  requirements. 

□  Early  test  strategy,  Test  and  Evaluation  Strategy 


Reference  Material:  AFI  99-103,  AFI  16-1001,  AFI  16-1002,  Defense  Acquisition  Guide  (DAG)  Chapter  9 
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Other  Considerations:  Work  products  can  be  developed  by  either  the  acquirer  or  suppliers.  They  should 
be  selected  based  on  their  contribution  to  meeting  project  objectives  and  requirements,  and  to  addressing 
project  risks. 


VG1P2  Identify  the  work  products  to  be  verified  and  the  verification  methods 
that  will  be  used 


Description:  Identify  work  products  requiring  verification  and  any  associated  constraints.  Develop, 
coordinate  and  document  verification  methods.  Develop  verification  methods  concurrently  and  iteratively 
with  the  product  and  product-component  requirements  and  designs. 

Typical  Work  Products: 


□  TEMP 

□  Verification  Methods  Table  (e.g.  Requirements  Specification  Section  4) 


Reference  Material:  Defense  Acquisition  Guide  (DAG)  Chapter  9  and  AFI  99-103  -  Capabilities-Based 
Test  &  Evaluation 

Other  Considerations:  Work  products  are  developed  by  either  the  acquirer  or  suppliers.  Assure 
verification  methods  address  the  technical  approach  to  work  product  verification  and  the  specific 
approaches  that  will  be  used  to  verify  that  work  products  meet  their  specified  requirements. 


VG1P3  Establish  and  maintain  the  environment  and  resources  needed  to 
support  verification 


Description:  Identify  facilities  and  support  required.  Ensure  that  impacts  to  external  environment  (e.g. 
emission  of  various  pollutants  and  noise  into  the  environment,  electromagnetic  radiation,  etc)  are  within 
allowable  NEPA  limits.  Obtain  required  certifications. 

Typical  Work  Products: 


□  TEMP 

□  PESHE 

□  Certifications  (e.g.,  safety,  hazard  classification,  insensitive  munitions  qualification,  SEEK  EAGLE, 
security,  airworthiness,  flightworthiness,  frequency  authorizations,  information  assurance) 

□  Statement  of  Capability  (SOC)  (Agreement  between  program  office  &  test  organization) 


Reference  Material: 

Other  Considerations:  Resources  include  labs,  assembly  areas,  operators,  tools,  automation,  training, 
maintenance  infrastructure,  test  ranges  and  simulation  environments. 


VG1P4  Establish  verification  procedures  and  criteria 


Description:  Develop,  coordinate  and  document  verification  criteria  and  procedures. 

Typical  Work  Products: 


□  Work  product  test  plan/procedures 
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Reference  Material: 
Other  Considerations: 


VG2  Peer  reviews  are  performed 

VG2P1  Prepare  for  peer  reviews  of  selected  work  products 


Description:  Ensure  peer  reviews  are  planned  on  selected  work  products  to  find  and  remove  defects  in 
order  to  comply  with  established  standards.  Identify  data  to  be  collected  and  entry  and  exit  criteria. 
Establish  a  process  to  track  findings  through  disposition. 

Typical  Work  Products: 


□  Solicitation  documents  (ex:  SOW) 

□  System  engineering  plans 

□  Peer  review  entry  and  exit  criteria 

□  Peer  review  process  plan 

□  Software  Development  Plans 


Reference  Material: 

Other  Considerations:  Planning  activities  should  include  identification  of  key  reviewers,  scheduling, 
preparing  and  updating  materials  such  as  checklists  and  review  criteria,  and  selection  of  work  products  for 
review.  A  peer  review  is  a  method  for  conducting  verification  of  work  products  during  their  development 
that  has  had  great  success  in  detecting  defects,  especially  in  documents  for  requirements  and  design.  The 
intent  is  for  the  project  to  use  peer  reviews  on  selected  products  to  find  and  remove  defects  and  to  ensure 
compliance  to  applicable  standards.  The  suppliers  typically  use  peer  reviews  internally  on  selected  interim 
work  products  during  development,  but  the  acquirer  should  not  rely  solely  on  these  results.  Peer  reviews 
within  a  project  should  include  "independent"  participants  from  other  organizations  not  directly  connected  to 
the  material  being  reviewed. 


VG2P2  Conduct  peer  reviews  on  selected  work  products  and  identify  issues 
resulting  from  the  peer  review 

Description:  Record  and  retain  results  of  peer  reviews,  including  defects,  issues,  and  action  items.  Assign 
action  items  and  communicate  issues  to  stakeholders. 

Typical  Work  Products: 

□  Findings 

□  Action  items 
Reference  Material: 

Other  Considerations: 


VG3  Work  products  are  verified 

VG3P1  Perform  verification  on  the  selected  work  products 
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Description:  Verify  work  products,  using  approved  methods  and  procedures,  to  promote  early  detection  of 
problems,  timely  removal  of  defects,  and  reduce  cost.  Conduct  Functional  Configuration  Audits  (FCA)  to 
verify  the  actual  performance  meets  specification  requirements.  Verify  that  interfaces  conform  to 
established  standards,  agreements  and  specifications,  and  lower  level  designs  are  consistent  with  top-level 
designs.  Implement  pedigree  checks  and/or  test  item  configuration  auditing  for  important  test  events. 

Typical  Work  Products: 


□  FCA  results 

□  Findings 

□  Action  items 

□  Test  report 


Reference  Material: 
Other  Considerations: 


VG3P2  Analyze  and  document  the  results  of  all  verification  activities 


Description:  Document  and  manage  the  results  of  verification  as  an  element  of  the  project  data 
management  effort.  Conduct  analysis  to  identify  trends,  process  improvement  opportunities  and  potential 
risks. 

Typical  Work  Products: 


□  Test  reports 


Reference  Material: 
Other  Considerations: 


VG3P3  Initiate  and  document  corrective  actions 


Description:  Track  corrective  actions  until  subsequent  verification  activities  confirm  compliance  with 
requirements. 

Typical  Work  Products: 

□  Action  item  list 

□  Test  report 


Reference  Material: 

Other  Considerations:  Data  on  verification  and  corrective  actions  should  be  available  to  the  acquirer  and 
suppliers  organizations  as  well  as  to  other  appropriate  stakeholders. 


VG4  Prepare  for  validation 

VG4P1  Develop  a  product  validation  strategy  and  identify  work  products  for 
validation. 


Version  2-21  September  2010 


72 


Version  2-21  September  2010 


Description:  Ensure  the  validation  strategy  addresses  the  technical  approach  to  demonstrate  that  user 
needs  are  satisfied.  Select  work  products  for  validation  against  the  users’  needs  and  to  address  product 
performance  risks. 

Typical  Work  Products: 


□  Product  validation  strategy  (briefing  charts,  outline,  etc) 


Reference  Material: 

Other  Considerations:  It  is  important  that  the  team  define  at  the  outset  the  degree  to  which  validation  is 
required  both  early  in  the  project  (e.g.,  requirements  validation  activities)  and  later  when  the  end  products 
are  available.  All  planned  modeling,  simulation,  ground  and  flight  test  activities  to  validate  performance, 
safety,  suitability,  survivability,  and  effectiveness  in  the  intended  operational  environment  is  documented. 


VG4P2  Establish  and  maintain  validation  criteria,  methods  and  procedures. 


Description:  Develop,  coordinate  and  document  validation  criteria,  methods  and  procedures.  Define 
validation  procedures  and  criteria  to  ensure  that  the  product  will  fulfill  its  intended  use  when  placed  in  its 
intended  environment.  Develop  and  document  procedures  and  criteria  to  support  military  utility 
assessments  by  the  operational  test  activity. 

Typical  Work  Products: 


□  Detailed  test  plans/procedures 


Reference  Material: 

Other  Considerations:  It  is  important  that  the  acquirer  define  at  the  outset  the  degree  to  which  validation 
is  required  both  early  in  the  project  (e.g.,  requirements  validation  activities)  and  later  when  the  end 
products  are  received. 


VG4P3  Establish  and  maintain  the  environment  and  resources  needed  to 
support  validation 


Description:  Ensure  that  an  adequate  environment  to  carry  out  validation  activities  is  maintained.  Ensure 
that  operators,  enabling  systems,  and  test  facilities  will  be  available  in  order  to  conduct  validation. 

Typical  Work  Products: 


□  Agreement  with  the  test  activity  (e.g.  Statement  of  capabilities  (SOC)) 

□  TEMP 


Reference  Material: 

Other  Considerations:  The  validation  environment  should  be  precisely  controlled  to  provide  for 

replication,  analysis  of  results,  and  revalidation  of  problem  areas. 


VG4P4  Ensure  appropriate  certifications  &  accreditations  have  been 
completed 


Description:  Plan  for  and  obtain  certifications  required  for  validation  activities  before  relevant  need 
date. 
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Typical  Work  Products: 


□  Certifications  (e.g.,  safety,  hazard  classification,  insensitive  munitions  qualification,  SEEK 
EAGLE,  security,  airworthiness,  flightworthiness,  frequency  authorizations,  information 
assurance) 

□  Project  Mgt  Plan 

□  SEP 


Reference  Material:  AFMAN  63-119 

Other  Considerations:  Certification  of  readiness  to  test  before  entering  a  formal  product  validation  phase 
should  be  accomplished  by  the  project  manager. 


VG4P5  Establish  and  maintain  a  documented  plan  for  validation 


Description:  Document  all  modeling,  simulation,  ground  and  flight  test  activities  planned  for  product 
validation. 

Typical  Work  Products: 


□  TEMP 

□  Modeling  and  simulation  plan 


Reference  Material: 

Other  Considerations:  Proactively  seek  opportunities  for  combined  testing  with  the  operational  test 
agencies  and  oversight  offices. 


VG5  Validate  product  to  ensure  that  it  will  be  safe,  suitable  and  effective  in  the 
intended  operating  environment 

VG5P1  Perform  validation  on  the  selected  products  and  product-components 

Description:  Perform  validation  activities  and  collect  resulting  data  according  to  the  approved  methods, 
procedures,  and  criteria. 

Typical  Work  Products: 

□  Test  Data 

□  Detailed  test  plan 

□  Test  procedures 

Reference  Material: 

Other  Considerations:  Product  interoperates  with  the  enabling  systems  that  will  be  present  in  the 
operational  environment 


VG5P2  Analyze  and  document  the  results  of  the  validation  activities 
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Description:  Document  and  manage  results  of  validation  as  an  element  of  the  project  data  management 
effort.  Conduct  analysis  to  identify  trends  and  process  improvement  opportunities.  Identify  issues  and 
initiate  and  document  corrective  actions.  Track  corrective  actions  to  closure  by  an  established  and 
documented  process  and  communicate  to  appropriate  stakeholders. 

Typical  Work  Products: 


□  Test  Reports 

□  Validation  issues 

□  Validation  deficiency  reports 

□  Procedure  change  requests 

Reference  Material: 

Other  Considerations: 


9.  Generic  Practices 

Generic  practices  are  practices  that  must  be  accomplished  for  each  process  area.  Within  the  AF 
SEAM,  goals  were  created  within  each  process  area  for  application  of  both  the  generic  practices 
and  the  specific  practices  that  appear  in  each  process  area  description.  Generic  practices 
improve  the  power  of  a  process  by  ensuring  that  the  specific  practices  are  executed  and  that 
there  is  appropriate  planning  of  the  process  to  ensure  that  it  is  feasible  and  well  supported  and 
that  stakeholders  are  properly  involved. 


9.1  GP1  Establish  and  maintain  the  description  of  a  process 


Description:  Establish  and  maintain  a  documented  description  of  the  process. 

Typical  Work  Products: 


□  Operational  instructions  (Ols) 

□  Process  descriptions 


Reference  Material: 

Other  Considerations:  The  documented  process  provides  the  basis  for  planning,  performing,  and 
managing  the  activities,  work  products,  and  services  associated  with  the  process.  The  process  description 
may  reference  or  tailor  higher  level  processes. 


9.2  GP2  Establish  and  maintain  plans  for  performing  the  process 


Description:  Establish  and  maintain  a  plan  to  implement  the  activities  necessary  to  perform  the  process. 
As  a  minimum,  a  plan  describes:  activities,  resources,  and  schedule  required  to  accomplish  the  process. 

Typical  Work  Products: 
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□  Process  execution  plans  as  applicable  to  the  process  area  (Ex:  SEP,  LCMP,  ILSP,  CMP,  and 
associated  internal  working  documents) 


Reference  Material: 

Other  Considerations:  Objectives  may  be  derived  from  other  plans  (e.g.,  the  project  plans)  and  should 
include  objectives  for  the  specific  situation,  such  as  quality,  cost,  and  schedule  objectives.  Schedules  may 
be  event  driven.  The  plan  for  performing  the  process  should  typically  include  the  following  elements: 

•  process  description 

•  standards  for  the  work  products  and  services  of  the  process 

•  requirements  for  the  work  products  and  services  of  the  process 

•  specific  objectives  for  the  performance  of  the  process  (e.g.,  quality,  time  scale,  cycle  time,  and 
resource  usage) 

•  dependencies  among  the  activities,  work  products,  and  services  of  the  process 

•  resources  (including  funding,  people,  and  tools)  needed  to  perform  the  process 

•  assignment  of  responsibility  and  authority 

•  training  needed  for  performing  and  supporting  the  process,  including  proficiency  and  project  team 
training 

•  work  products  to  be  placed  under  configuration  management  and  the  level  of  configuration 
management  for  each  item 

•  measurement  requirements  to  provide  insight  into  the  performance  of  the  process,  its  work  products, 
and  its  services 

•  involvement  of  identified  stakeholders 

•  activities  for  monitoring  and  controlling  the  process 

•  objective  evaluation  activities  for  the  process  and  the  work  products 

•  management  review  activities  for  the  process  and  the  work  products 

Establishing  a  plan  includes  documenting  the  plan  and  providing  a  description  of  the  process  it  implements. 
Maintaining  the  plan  includes  changing  it  as  necessary  in  response  to  either  corrective  actions  or  to  changes  in 
requirements  and  objectives  for  the  process. 


9.3  GP3  Provide  adequate  resources  for  performing  the  process,  developing 
the  work  products,  and  providing  the  services  of  the  process 


Description:  Ensure  that  the  resources  necessary  to  perform  the  process  are  available  when  they  are 
needed.  Resources  include  adequate  funding,  appropriate  physical  facilities,  skilled  people,  and 
appropriate  tools. 

Typical  Work  Products: 
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□  Staffing  plan  (names  and  skill  sets) 

□  Funding  profile 

□  Resource  availability  schedule 

□  Tools  list 

Reference  Material: 

Other  Considerations: 


9.4  GP4  Assign  responsibility  and  authority  for  performing  the  process, 
developing  the  work  products,  and  providing  the  services  of  the  process 


Description:  Ensure  that  there  is  accountability  for  performing  the  process  and  achieving  the  specified 
results  throughout  the  life  of  the  process.  Assign  responsibility  using  detailed  job  descriptions  or  project 
documents.  Ensure  personnel  designated  have  the  documented  authority  to  perform  the  assigned 
responsibilities. 

Typical  Work  Products: 


□  Roles  and  responsibility  descriptions 

□  Charters 

□  Delegation  of  authority  letters/memo 


Reference  Material: 
Other  Considerations: 


9.5  GP5  Train  the  people  performing  or  supporting  the  process  as  needed. 


Description:.  Ensure  that  proficiency  and  process  training  for  the  entire  project  team  including 
stakeholders  is  performed  and  documented.  Overview  training  is  provided  to  orient  people  who  interact 
with  those  performing  the  work.  Allow  the  people  to  take  the  required  training. 

Typical  Work  Products: 


□  Training  materials 

□  Training  records 

□  Training  schedules 


Reference  Material: 

Other  Considerations  Examples  of  methods  for  providing  training  include  self-study;  self-directed  training; 
self-paced,  programmed  instruction;  formalized  on-the-job  training;  mentoring;  and  formal  and  classroom 
training. 
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9.6  GP6  Monitor  and  control  the  process 


Description:  (1)  Ensure  that  the  process  is  performed  as  planned.  (2)  Objectively  ensure  that  the  process 
adheres  to  its  process  description.  (3)  Objectively  evaluate  work  products  against  the  applicable  standards 
and  ensure  the  process  yields  the  desired  outcomes/products.  (4)  Identify  issues  and  take  corrective 
action  as  necessary. 

Typical  Work  Products: 


□  Process  review  reports 

□  Metrics 

□  Deficiency  reports 

□  Corrective  action  plans 


Reference  Material: 

Other  Considerations:  This  typically  involves:  (A)  Measuring  attributes  of  the  process  and  work  products 
produced  by  the  process;  (B)  Provide  metrics  showing  trends  to  support  proactive  improvements;  (C) 
Developing  corrective  actions  and  track  to  closure;  and  (D)  Providing  feedback  to  process  stakeholders. 
Additionally,  independent  assessments  and  feedback  are  an  effective  means  of  ensuring  compliance  and 
continuous  process  improvement.  NOTES:  1.  Under  item  “(2)”  above  the  word  “objectively”  refers  to  an 
examination  of  the  process  which  may  be  performed  by  peers  or  as  an  integral  part  of  the  process.  2. 
Under  item  “(3)”  above  the  word  “objectively”  refers  to  ensuring  that  the  process  meets  customer  needs 
and  may  involve  direct  customer  feedback. 


9.7  GP7  Review  the  activities,  status,  and  results  of  the  process  with  higher 
level  management  and  resolve  issues 


Description:  Provide  higher  level  management  with  visibility  into  the  results  of  execution  of  the  process. 
Perform  process  reviews  to  provide  recommendations  for  policy,  process,  and  resources  to  facilitate 
improvement. 

Typical  Work  Products: 

□  Briefings 

□  Meeting  minutes 

□  Recommendations 

Reference  Material: 

Other  Considerations: 
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10.  Point  of  Contact 

AF  SEAM  is  designed  as  a  Continuous  Process  Improvement  (CPI)  tool.  Accordingly, 
the  model  itself  must  undergo  CPI  in  order  to  remain  current  and  relevant.  Please  submit 
recommended  improvements  or  comments  to  your  center’s  AF  SEAM  POC. 
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